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FOREWORD 


The  ARI  Fort  Knox  Field  Unit  has  been  involved  for  approximately 
ten  years  in  the  development  of  innovative  approaches  to  training  for 
the  Armor  community  and  for  the  Army  as  a  whole.  Just  recently,  this 
effort  has  been  given  special  emphasis  through  formation  of  the  Training 
Technology  Field  Activity  (TTFA),  a  partnership  among  ARI,  the  Training 
and  Doctrine  Command,  and  the  U.S.  Army  Armor  Center  and  School.  The 
purpose  of  the  TTFA  is  to  increase  the  effectiveness  and  efficiency  of 
training  through  the  application  of  appropriate  new  technologies. 

Initial  efforts  of  the  Fort  Knox  TTFA  have  been  concentrated  upon 
the  institutional  program  for  training  Ml  tank  commanders.  Before 
introducing  new  technologies  into  the  training  program,  it  was  necessary 
to  ensure  that  the  appropriate  groundwork  had  been  accomplished  in  terms 
of  instructional  analysis,  design,  and  development.  The  first  report  on 
this  work  presented  the  results  of  the  analysis  phase  by  providing  a 
review  and  supplement  of  available  job  and  tasks  analyses  for  the  Ml 
tank  commander  duty  position.  The  second  report  built  upon  the  analysis 
phase  by  presenting  a  general  design  for  an  Ml  tank  commander  training 
program.  Other  reports  currently  in  preparation  present  selected  train¬ 
ing  and  evaluation  products  for  the  training  program  as  well  as  an 
example  instruction  on  decision  making  and  problem  solving  tasks. 

The  present  report  is  a  guidebook  for  the  analysis  and  design 
phases  of  course  revision.  The  procedures  presented  in  the  guidebook 
are  based  on  those  used  to  revise  the  training  program  for  Ml  tank 
commanders.  Illustrations  of  procedures  are  taken  from  that  course 
revision  process.  The  guidebook  is  intended  to  provide  a  useful  "how 
to"  reference  for  revision  of  future  training  programs. 


EDGAR  M.  JOHNSON 
Technical  Director 
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CHAPTER  1 
INTRODUCTION 


1-1.  Background.  The  present  job  aid  was  developed  as  a  part  of  a 
project  in  which  the  Systems  Approach  to  Training  (TRADOC  Regulation 
350-7)  was  applied  to  revising  the  Basic  Noncommissioned  Officers  Course 
for  Ml  tank  commanders  (19K  BNCOC) .  The  purpose  of  the  revision  was  to 
incorporate  state-of-the-art  training  technologies  into  19K  BNCOC.  At 
the  outset  of  the  project,  it  was  recognized  that  the  procedures  for  the 
Systems  Approach  to  Training  (SAT)  were  sometimes  ill-defined  and 
inappropriate  to  particular  situations.  One  of  the  purposes  of  the 
project  was  to  make  appropriate  elaborations  of  and  modifications  to  SAT 
and  try  them  out  in  revising  19K  BNCOC.  The  procedures  used  to  revise 
19K  BNCOC  are  documented  in  the  present  job  aid,  which  is  designed  as  a 
guide  to  future  course  revision  projects. 

1-2.  The  State  of  Instructional  Systems  Development. 

a.  SAT  is  but  one  example  of  a  class  of  procedures  under  the  more 
general  rubric  of  instructional  systems  development  (ISD) .  Though 
differing  in  the  procedural  details,  all  ISD  models  are  cast  within  a 
systems  framework  and  have  the  following  general  characteristics: 

(1)  The  content  of  training  is  specified  in  terms  of  training 
objectives  that  are  derived  from  an  analysis  of  the  student’s  prospec¬ 
tive  job.  Each  training  objective  is  a  precise  statement  of  the 
conditions,  actions,  and  standards  for  performing  a  particular  job  task 
or  group  of  tasks. 

(2)  Performance  criteria  are  developed  to  provide  evidence 
that  a  student  has  achieved  a  particular  training  objective.  It  is 
significant  to  note  that  in  ISD  models  performance  criteria  are  derived 
directly  from  the  training  objectives  rather  than  from  existing  instruc¬ 
tion. 


(3)  Methods  of  instruction  are  devised  to  help  students 
attain  the  training  objectives  in  an  efficient  manner.  In  other  words, 
the  purpose  of  instruction  is  to  maximize  student  performance  on  the 
specified  criteria. 

(A)  The  training  system  is  continuously  evaluated  and  modi¬ 
fied.  If,  for  instance,  a  particular  training  approach  is  not  effective 
in  producing  acceptable  performance  levels,  the  approach  is  changed  and 
the  effect  on  performance  is  measured.  This  principle  of  iterative 
evaluation  and  modification  extends  to  other  components  of  the  instruc¬ 
tional  system  as  well. 


b.  It  is  important  to  emphasize  that,  although  ISD  models  share 
some  common  characteristics,  there  is  no  single  universally  accepted  set 
of  procedures  for  carrying  out  instructional  development.  In  fact, 
Goldstein  (1980)  commented  that  "there  are  almost  as  many  systems 
approaches  as  there  are  authors  on  the  subject"  (p.  231).  The  prolifer¬ 
ation  of  procedures  is  not  necessarily  a  negative  aspect  of  ISD  given 
the  inexactness  of  instructional  science  and  the  systems  principle  of 
iterative  evaluation  and  modification.  Although  the  present  guide  does 
not  present  a  formal  review  of  other  approaches  to  ISD,  there  is  an 
outline  of  the  similarities  and  differences  between  the  present  approach 
and  other  relevant  approaches  in  an  appendix  to  this  guide. 

1-3.  Scope  of  the  Present  Guide.  Some  of  the  limitations,  applica¬ 
tions,  emphases,  and  other  general  characteristics  of  the  present  guide 
are  described  below. 

a .  Limited  to  Course  Revision. 

(1)  The  present  guide  is  designed  to  apply  to  revising  an 
existing  course  of  instruction  rather  than  creating  one  anew.  This 
should  not  be  a  serious  limitation  in  that  most  training  development 
projects  are  actually  revisions  of  existing  courses.  The  most  probable 
reason  for  revising  a  course  is  the  fielding  of  new  military  equipment 
that  is  related  to  a  particular  course  of  instruction.  The  recent  tank 
force  modernization  project  to  replace  M60  series  tanks  with  Ml  tanks  is 
a  case  in  point.  In  such  cases,  the  new  equipment  replaces  older  equip¬ 
ment  that  is  similar  in  function  and/or  operation.  Therefore,  much  of 
the  task  documentation  and  other  materials  from  the  old  course  are  still 
relevant  to  some  extent  to  the  revised  course, 

(2)  Other  approaches  to  ISD  have  provisions  for  incorporating 
existing  training  materials  into  a  newly  developed  course  of  instruc¬ 
tion.  However,  they  typically  ignore  existing  documentation  from  the 
earlier  analysis  and  design  phases.  The  effects  of  integrating  this 
sort  of  valuable  documentation  into  the  revision  process  whenever  appro¬ 
priate  is  to  save  a  significant  amount  of  course  development  time  and  to 
prevent  the  developer  from  "reinventing  the  wheel." 

b.  Applies  to  Only  Analysis  and  Design  Phases.  SAT  procedures 
are  divided  into  five  phases  of  development  process:  evaluation, 
analysis,  design,  development,  and  implementation.  The  present  guide 
applies  only  to  the  analysis  and  design  phases,  typically  the  first  two 
steps  in  the  revision  process.  This  division  of  the  revision  process  is 
significant  in  that  the  first  two  phases  can  be  accomplished  by  the 
training  developer/reviser  himself,  perhaps  with  the  assistance  of  a 
small  staff.  The  work  involved  in  later  phases  is  typically  parcelled 
out  to  other  personnel  and  organizations:  development  to  the  Direc¬ 
torate  of  Training  Development  (DOTD)  and/or  to  an  instructional 
department  (e.g.,  Weapons  Department,  Maintenance  Department,  etc.); 
implementation  to  the  instructional  staff  of  the  existing  course;  and 
evaluation  to  the  Directorate  of  Evaluation  and  Standardization  (DOES). 
In  a  sense  then,  the  present  guide  is  designed  to  get  the  training 
developer  started  revising  the  course  before  he  hands  off  parts  of  the 
process  to  others. 
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c.  Emphasis  on  the  "How  To”  of  Course  Revision.  One  notable 
aspect  of  SAT  is  its  overemphasis  of  the  "what"  of  training  development 
at  the  expense  of  the  "how  to."  That  is.  Regulation  350-7  lists  a 
series  of  minimum  essential  elements  (products  of  various  analyses)  but 
fails  to  specify  procedures  for  developing  those  elements.  In  general, 
the  regulation  refers  the  developer  to  other  related  documents  for 
procedural  details.  Most  important  of  these  supporting  documents  is  the 
Interservice  Procedures  for  Instructional  Systems  Development  (TRADOC 
Pamphlet  350-30)  herein  referred  to  as  IPISD.  Although  many  procedures 
for  SAT  products  are  given  in  IPISD,  some  are  not.  In  contrast  to  the 
SAT  document  (Regulation  350-7),  the  emphasis  of  the  present  guide  is  on 
the  "how  to"  (i.e.,  the  procedures)  of  course  revision.  Examples  from 
the  19K  BNCOC  project  are  presented  to  illustrate  many  of  the  procedures 
described  in  the  text. 

d.  Emphasis  on  the  "Doable."  Many  of  the  procedures  cited  in 
IPISD  are  certainly  valid,  but  they  are  impractical  given  the  time  and 
resource  constraints  facing  a  typical  course  developer/reviser .  An 
important  criterion  for  in  luding  a  particular  procedure  in  the  present 
guide  was  that  it  appeared  "doable"  within  such  constraints. 

e.  Based  on  19K  BNCOC  and  Other  Sources.  Related  to  the  pre¬ 
ceding  point,  one  of  the  advantages  of  the  present  procedures  compared 
to  other  guides  to  training  development  is  that  they  are  based  on  a 
"real  world"  application  of  SAT  rather  than  on  an  abstract  conceptuali¬ 
zation  of  how  training  should  be  developed.  On  the  other  hand,  results 
from  the  field  tryout  indicated  that  the  proposed  procedures  were  not 
always  complete  and/or  practical.  In  order  to  present  a  useful  job  aid, 
it  was  necessary  to  supplement  the  procedures  used  in  redesigning  19K 
BNCOC  with  procedures  proposed  in  other  sources.  Not  all  of  these 
sources  are  referenced  in  the  text.  They  are  nevertheless  included  in 
the  Bibliography  at  the  end  of  the  guide  in  order  to  provide  additional 
resources  that  could  be  useful  to  the  course  developer/reviser. 

f.  Emphasis  on  Interactions.  Although  most  of  the  actions 
required  in  the  analysis  and  design  phases  of  course  revision  can  be 
accomplished  by  a  single  individual,  this  does  not  mean  that  he  or  she 
should  work  in  a  vacuum.  The  course  reviser  should  constantly  be  In 
touch  with  both  subject  matter  experts  (SMEs)  for  technical  advice  and 
his  supervisors  for  decisions  on  training  management  matters.  The 
appropriate  points  for  such  interactions  are  identified  in  the  present 
procedures. 


g.  Eight  Major  Activities.  The  process  of  course  revision  is 
organized  around  eight  major  activities  or  phases.  The  activities  are 
named  to  convey  the  output  of  each  phase  rather  than  describe  the  action 
involved.  For  example,  the  SAT  activity  named  "Analyze  Tasks"  is  herein 
referred  to  as  "Revise/Develop  Training  Objectives."  While  it  is  true 
that  this  phase  involves  analysis  of  job  tasks,  the  outcome  of  the 
analysis  (i.e.,  the  training  objectives)  is  not  immediately  obvious, 
especially  to  the  novice  course  developer/reviser .  The  correspondence 
of  these  activities  to  the  components  of  other  ISD  models  is  shown  in 
the  Appendix.  The  following  eight  activities  are  described  in  detail  in 
the  subsequent  chapters  of  the  guide: 


(1)  Examine/Revise  Course  Objective 

(2)  Describe  Course  Entrants 

(3)  Revise  Task  Inventory 

(4)  Select  Tasks  for  Training 

(5)  Revise/Develop  Training  Objectives 

(6)  Organize  Training  Objectives 

(7)  Develop  Tests 

(8)  Outline  Training  Program 


CHAPTER  2 


EXAMINE/REVISE  COURSE  OBJECTIVE 

2-1.  Introduction.  Before  beginning  the  process  of  course  revision, 
you  must  closely  examine  the  overall  purpose  of  the  course.  In  accord¬ 
ance  with  current  ISD  models,  the  process  of  course  revision  is  viewed 
as  a  system.  The  course  objective  provides  an  overall  goal  for  the 
process  thereby  providing  a  unifying  function  in  the  system. 

2-2.  Identify  Need  for  Revision.  Make  sure  you  understand  the  reason 
for  revising  the  course.  Example  reasons  include  the  introduction  of 
new  equipment  or  the  reorganization  of  job  functions.  The  exact  need 
for  revision  should  be  discussed  at  length  with  training  management.  A 
memorandum  may  be  needed  to  clarify  the  understanding  between  you  and 
training  management. 

2-3.  Examine  the  Present  Course  Objective.  A  statement  of  the  present 
course  objective  can  be  found  in  the  Freface  of  the  Program  of  Instruc¬ 
tion  (POI) .  Examine  the  adequacy  of  the  objective  with  respect  to  the 
identified  need  for  revision.  The  basic  question  is  whether  or  not  the 
statement  accurately  describes  the  objective  of  the  revised  course  of 
instruction.  You  may  wish  to  discuss  this  issue  with  personnel  who  are 
knowledgeable  about  the  present  course.  Such  knowledgeable  people 
include  course  developers,  instructors,  course  graduates,  etc.  Be  sure 
to  keep  these  discussions  at  a  very  general  level  and  not  at  the  level 
of  adding  or  deleting  specific  tasks. 

2-U.  Modify  Course  Objective.  If  the  present  course  objective  is  inad¬ 
equate,  it  should,  of  course,  be  appropriately  modified.  As  general 
guidelines,  the  course  objective  should  include  the  following  elements: 

a.  Who  Is  to  Be  Trained.  The  course  objective  should  include  the 
type  of  personnel  being  trained,  i.e.,  enlisted  personnel,  noncommis¬ 
sioned  officers,  officers,  etc. 

b.  What  Is  to  Be  Trained.  Include  a  general  statement  of  what 
personnel  are  being  trained  to  do.  If  the  statement  is  not  sufficiently 
clear  you  can  include  the  sorts  of  tasks  that  the  job  incumbent  will  be 
expected  to  perform.  Examples  include: 

(1)  to  perform  maintenance  on  a  tracked  vehicle, 

(2)  to  drive  an  Ml  tank,  or 

(3)  lead  a  platoon. 

c.  The  Result  of  Training.  Describe  the  job  a  course  graduate 
can  perform  as  a  result  of  training  and  the  associated  level  of  compe¬ 
tency  (familiarized,  qualified,  distinguished).  Examples  include: 
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(1)  a  distinguished  tank  gunner, 

(2)  a  qualified  platoon  sergeant,  or 

(3)  a  familiarized  turret  mechanic. 

2-5.  Example  Course  Objective.  The  adequacy  of  the  present  course 
objective  for  19K  BNCOC  was  examined.  The  existing  course  objective  did 
not  include  some  of  the  elements  discussed  above.  A  suggested  course 
objective  might  be: 

The  objective  of  19K  BNCOC  is  to  train  noncommissioned 
officers  to  command  an  Ml  tank.  Included  in  the  course 
are  tasks  related  to  operating  and  maintaining  the  Ml 
tank  commander's  station  as  well  as  tasks  concerned  with 
the  leadership,  training,  and  direction  of  the  tank's 
crew.  The  course  graduate  will  be  fully  qualified  to 
command  an  Ml  tank. 

2-6.  Have  Revised  Objective  Approved.  Submit  in  writing  any  revision 
of  the  course  objective  to  training  management  for  their  approval. 
Emphasize  the  importance  of  course  objective  as  a  statement  of  the  basic 
parameters  to  course  revision.  For  that  reason,  secure  management's 
approval  of  the  objective  before  proceeding  with  course  revision. 


CHAPTER  3 


DESCRIBE  COURSE  ENTRANTS 


3-1.  Introduction. 


a.  In  order  to  develop  an  appropriate  course  of  instruction,  you 
will  need  to  determine  the  relevant  skills,  knowledges,  and  aptitudes 
that  students  possess  prior  to  entering  the  revised  course.  According 
to  IPISD,  this  determination  is  accomplished  by  obtaining  a  sample  that 
is  representative  of  course  entrants,  and  then  measuring  their  per¬ 
formance  on  tests  derived  from  an  analysis  of  the  training  objectives 
(Chapter  8).  Training  objectives  are  then  adjusted  in  accordance  with 
soldier  performance.  The  results  from  a  field  study  of  ISD  users 
(Vineberg  &  Joyner,  1980)  indicated  that  such  procedures  were  not  used 
in  development  of  actual  courses.  These  investigators  concluded  that 
the  advantages  of  having  this  sort  of  information  about  course  entrants 
are  probably  outweighed  by  the  time  and  expense  Involved  in  obtaining 
appropriate  subjects  and  in  measuring  performance  prior  to  actual 
implementation  of  the  course. 

b.  In  the  present  section,  a  modified  set  of  procedures  is  pre¬ 
sented  for  describing  course  entrants.  The  data  collection  requirements 
of  the  IPISD  are  eliminated  by  restricting  the  analysis  to  data  that  are 
a  matter  of  record.  One  possible  source  of  information  about  course 
entrants  may  be  obtained  from  the  results  of  any  pre-course  tests. 
Another  source  is  biographical  information  about  a  course  entrant's 
experience  and  ability  level.  The  purpose  of  the  modified  procedure  is 
to  collect  all  available  information  about  course  entrants  that  might  be 
relevant  to  the  course  revision  process. 

c.  For  the  most  part,  there  is  a  sequential  logic  to  the  phases 
of  course  revision.  This  is  because  the  products  of  one  phase  serve  as 
input  to  the  next  phase.  In  contrast,  the  location  of  the  present  phase 
(Describe  Course  Entrants)  in  the  sequence  is  not  immediately  obvious 
for  a  couple  of  reasons.  First,  while  helpful,  the  output  of  this  phase 
is  not  a  requirement  for  any  other  phase  in  course  revision.  Conse¬ 
quently,  this  phase  may  be  bypassed  if  appropriate  data  are  not 
available.  Second,  if  data  are  available,  this  phase  can  potentially 
provide  input  to  a  number  of  other  phases.  Because  of  this  potential 
impact,  the  analysis  of  course  entrants  should  be  accomplished  as  early 
as  possible  in  the  course  revision  process  if  the  relevant  data  are 
available . 

3-2.  Obtain  Student  Data.  Some  sort  of  student  records  should  be  main¬ 
tained  by  the  faculty  of  the  existing  course.  Request  permission  to 
take  a  quick  inventory  of  all  the  available  information  on  students  who 
have  enrolled  in  the  course.  There  are  essentially  two  types  of  data: 
biographical  data  on  student  background  and  abilities,  and  performance 
data  from  various  precourse  tests  that  are  presently  being  administered. 
Record  any  information  that  might  be  relevant  to  student  performance  in 
the  course.  Be  as  inclusive  as  possible  in  obtaining  data.  It  is  some¬ 
times  difficult  to  foresee  the  significance  of  the  data  until  after  they 
have  been  collected  and  analyzed. 
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3-3.  Analyze  Biographical  Data.  Biographical  data  Include  Information 
such  as  age,  number  of  years  in  service,  highest  formal  military  train¬ 
ing,  last  SQT  score,  etc.  These  data  are  sometimes  obtained  by  means  of 
questionnaires  administered  during  in-processing  for  the  present  course. 
The  following  are  some  suggested  methods  for  analyzing  and  interpreting 
the  data. 

a.  Construct  Frequency  Distributions.  A  standard  statistical 
method  for  summarizing  data  is  to  construct  a  frequency  distribution.  A 
frequency  distribution  is  a  table  that  lists  all  values  of  a  variable 
along  with  the  corresponding  frequencies  of  occurrence.  Biographical 
data  may  be  either  quantitative  or  qualitative.  Quantitative  informa¬ 
tion  is  where  the  student  information  is  expressed  in  numbers,  e.g.,  age 
in  years,  SQT  scores,  etc.  In  contrast,  qualitative  data  is  where 
student  responses  are  not  numerically  related.  Examples  of  qualitative 
data  include  highest  formal  military  training,  parent  unit,  present  job, 
etc.  The  procedures  for  constructing  frequency  distribution  differ  for 
quantitative  and  qualitative  data. 

(1)  Qualitative  Data.  The  procedures  for  constructing  fre¬ 
quency  distributions  for  these  data  are  fairly  straightforward. 

(a)  Determine  the  possible  range  of  responses  to  a 
qualitative  question.  Using  19K  BNCOC  as  an  example,  course  entrants 
were  asked  to  name  their  current  job.  The  respondents  gave  four  dif¬ 
ferent  answers:  tank  commander,  gunner,  driver,  and  instructor. 

(b)  Next,  determine  the  number  of  soldiers  who  gave  each 
sort  of  answer.  Of  the  38  soldiers  who  gave  interpretable  answers  to 
the  example  question,  24  were  tank  commanders,  5  were  gunners,  8  were 
instructors,  and  1  was  a  driver. 

(c)  Construct  a  table  which  lists  each  response  along 
with  the  corresponding  number  or  frequency  of  soldiers  giving  each  sort 
of  response.  An  example  distribution  is  given  in  Figure  3-1  that  lists 
the  results  from  the  question  about  entrants'  current  job.  As  in  the 
current  example,  you  may  wish  to  include  a  column  for  the  total  number 
of  responses  in  order  to  determine  the  relative  proportion  of  responses 
in  each  category. 

(2)  Quantitative  data. 

(a)  Frequency  distributions  for  quantitative  data  are 
similarly  constructed.  However,  the  number  of  possible  responses  may  be 
very  great  for  a  quantitative  variable.  For  example,  a  quantitative 
variable  such  as  weight  could  potentially  have  an  infinite  number  of 
possible  responses  depending  on  the  level  of  precision  of  the  measuring 
scale.  For  that  reason,  the  frequency  distribution  table  often  lists 
classes  of  responses  instead  of  individual  responses.  The  following  are 
some  standard  statistical  conventions  that  are  used  to  derive  appropr¬ 
iate  classes  of  quantitative  responses: 


Present  Job 


Frequency 


Tank  Commander 

24 

Gunner 

5 

Driver 

1 

Instructor 

_8 

Total 

38 

Figure  3-1.  Example  Frequency  Distribution  for  Qualitative  Data 


JL^  All  classes  must  have  an  equal  range  of 
responses.  This  range  is  usually  referred  to  as  the  interval  size. 

2^  For  larger  samples  (i.e.,  greater  than  50 
soldiers)  there  should  be  between  8  to  15  intervals.  However,  the 
number  of  intervals  may  be  somewhat  smaller  than  8  for  smaller  samples. 

_3  The  interval  size  should  be  either  2,  3,  5,  or  a 

multiple  of  5. 

4^  The  lowest  score  in  each  interval  should  be  an 
even  multiple  of  the  interval  size. 

(b)  An  example  of  a  quantitative  frequency  distribution 
is  presented  in  Figure  3-2.  In  this  example,  19K  BNCOC  entrants  were 
asked  to  report  the  number  of  years  they  had  been  on  active  duty.  After 
experimenting  with  several  interval  sizes,  an  interval  size  of  two  was 
determined  to  be  most  appropriate  for  this  relatively  small  data  set. 
With  an  interval  size  of  two,  a  total  of  six  class  intervals  were  needed 
to  cover  the  entire  range  of  data.  Note  that  each  of  the  intervals 

started  with  an  even  multiple  of  the  interval  size  (i.e.,  two). 


Years  of  Active  Duty 

Frequency 

12  -  13 

1 

10  -  11 

2 

8-9 

6 

6-7 

9 

4-5 

15 

2-3 

_4 

Total 

37 

Figure  3-2.  Example  Frequency  Distribution  for  Quantitative  Data 


.  Describe  the  Typical  Entrant. 


(1)  Identify  the  central  tendency  of  the  frequency  distribu¬ 
tion  in  order  to  describe  the  background  of  the  typical  course  entrant. 
The  central  tendency  of  a  distribution  describes  the  response  value 
around  which  the  scores  are  centered.  The  most  useful  measure  in  this 
regard  is  the  mode,  which  is  defined  as  the  value  (or  values)  having  the 
greatest  frequency  and,  therefore,  the  highest  probability  of  occur¬ 
rence.  The  mode  is  also  the  most  appropriate  measure  of  central 
tendency  because  it  may  be  used  to  describe  qualitative  as  well  as 
quantitative  data.  Using  the  two  previously  presented  frequency  distri¬ 
butions  as  examples  (Figures  3-1  and  3-2),  we  may  conclude  that  the 
typical  19K  BNCOC  is  currently  a  tank  commander  with  4-5  years  of  active 
duty  in  the  Army. 

(2)  If  the  data  are  extremely  dispersed,  the  mode  may  not  be 
a  good  representation  of  the  central  tendency.  Dispersion  refers  to  the 
degree  to  which  the  scores  are  scattered  apart  from  one  another. 

Consider  the  following  example  taken  from  the  revision  of  19K  BNCOC 
(Figure  3-3).  The  scores  in  the  example  distribution  are  so  dispersed 
that  there  is  no  clear  central  tendency,  i.e.,  the  mode  does  not  provide 
an  adequate  description  of  the  data.  In  a  case  such  as  this,  you  should 
reexamine  the  distribution  as  a  whole  to  see  if  the  data  can  be 
restructured  to  provide  some  other  reasonable  description  of  the 
distribution.  In  the  present  example,  it  may  be  seen  that  whereas  there 
is  no  evident  mode,  a  large  majority  (79%)  of  the  tank  commanders  have 
less  than  18  months  on  the  job.  This  fact  seems  to  be  an  important 
qualification  to  the  previous  assertion  that  the  typical  19K  BNCOC 
entrant  is  already  a  tank  commander.  That  is,  while  the  typical  entrant 
is  presently  a  tank  commander,  he  has  not  served  in  that  capacity  for 
very  long  (less  than  1.5  years). 


Months  on  Job  as  TC 

Frequency 

36  -  41 

2 

30  -  45 

0 

24  -  29 

1 

28  -  23 

2 

12  -  17 

6 

6-11 

7 

0-5 
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Total 

24 

Figure  3-3.  Example  of  Highly  Dispersed  Data 


3-4.  Analyze  Performance  Data.  There  are  two  types  of  precourse  tests 
that  may  be  administered  to  course  entrants.  The  first  type  is  an  entry 
test,  which  measures  performance  on  skills  and  knowledges  that  are 
prerequisites  to  the  content  of  the  current  course.  If  an  entrant  fails 
an  entry  test  item,  he  may  be  denied  entry  into  the  course  or  he  may  be 
given  remedial  training  before  he  starts  the  course.  The  second  type  of 
precourse  test  is  referred  to  as  a  pretest,  which  is  a  test  on  the 
objectives  of  the  present  course.  An  entrant  who  passes  an  item  on  a 
pretest  may  then  skip  the  corresponding  item,  segment,  or  block  of 
training . 


a.  Calculate  the  NO  GO  Rate.  One  of  the  simplest  measures  of 

performance  on  a  precourse  test  item  is  the  NO  GO  rate.  The  NO  GO  rate 

is  defined  as  the  proportion  or  percent  of  entrants  failing  a  particular 
item.  For  instance,  if  17  entrants  out  of  a  total  sample  of  83  fail  a 

particular  item,  the  NO  GO  rate  is  17/83  =  .20  or  20%. 

b.  Identify  Performance  Problems.  Rank  order  the  test  items  from 
the  lowest  to  the  highest  NO  GO  rates.  Extremely  high  (say,  greater 
than  35%  on  an  entry  test  )  or  low  (less  than  5%  on  a  pretest)  NO  GO 
rates  should  be  noted  for  interpretation. 

3-5.  Interpret  the  Data.  The  data  gathered  in  this  phase  are  necessar¬ 
ily  limited  to  only  the  questions  asked  prior  to  the  course  or  to  the 
tasks  covered  in  a  precourse  performance  test.  No  conclusions  can  be 
drawn  about  matters  that  are  not  addressed  by  the  data.  Nevertheless, 
the  following  examples  provide  some  ideas  on  how  existing  data  might  be 
interpreted . 

a.  Biographical  Information. 

(1)  Examine  the  entrants'  background  and  experience  to  see  if 
they  match  the  external  requirements  as  stated  In  the  course  objective. 
If  there  are  serious  mismatches,  you  may  want  to  modify  the  require¬ 
ments.  Failing  that,  you  may  have  to  modify  the  course  objective  to 
correspond  with  the  reality  of  the  situation. 

(2)  Take  a  look  at  the  entrants'  experience  with  the  subject 
of  the  course.  If  students  have  no  experience  (i.e.,  they  are  being 
trained  on  a  new  vehicle),  you  will  need  to  consider  the  possibility  of 
providing  appropriate  orientation. 

(3)  Look  at  standardized  test  performance  (e.g.,  the  GT  com¬ 
ponent  of  the  ASVAB)  and  civilian  educational  level  to  predict  whether 
or  not  entrants  will  need  remediation  on  basic  skills.  You  may  have  to 
provide  for  concurrent  remediation  through  your  local  education  center. 

b.  Performance  Data.  The  interpretation  of  the  performance  data 
depends  largely  on  the  nature  of  the  precourse  test. 

(1)  For  entry  tests,  you  would  expect  a  fairly  low  NO  GO 
rate.  High  NO  GO  rates  for  an  entry  test  would  be  an  item  of  concern 
because  it  indicates  that  the  course  objectives  assume  a  higher  level  of 
skills  and  knowledge  than  entering  students  actually  possess.  As  a 
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result  you  might  recommend  that  the  previous  prerequisite  skills  and 
knowledges  be  adopted  as  training  objectives  in  the  revised  course, 

(2)  In  contrast,  you  would  expect  a  fairly  high  NO  GO  rate 
for  items  on  a  pretest.  Low  NO  GO  rates  on  a  pretest  would  mean  that 
students  already  possess  some  of  the  skills  and  knowledges  that  the 
course  is  intended  to  convey.  These  objectives  should  either  be  dropped 
from  the  course  or  only  used  as  entry  test  items. 
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CHAPTER  4 


REVISE  TASK  INVENTORY 


4-1.  Introduction.  For  purposes  of  analysis,  a  job  consists  of  an 
inventory  of  Individual  units  of  work  called  tasks.  In  the  present 
guide,  the  task  "inventory"  refers  to  the  tasks  that  are  to  be  trained 
in  a  particular  course,  not  necessarily  all  tasks  that  are  performed  on 
a  job.  In  this  phase  of  course  revision,  you  will  review  the  tasks  in 
the  current  inventory  with  respect  to  the  course  objective  and  revise 
the  inventory  appropriately. 

4-2.  Review  Procedural  Tasks.  Most  of  the  tasks  in  the  current 
inventory  are  procedural  in  nature.  That  is,  they  are  performed  in  a 
specific  sequence  of  steps  and  can  be  readily  trained  and  practiced. 

This  list  of  tasks  may  be  in  need  of  change.  Some  relevant  procedural 
tasks  may  have  been  omitted,  and  some  tasks  may  no  longer  belong  on  the 
revised  list.  Also,  some  tasks  may  be  improperly  stated.  Therefore,  it 
is  necessary  to  examine  the  existing  task  list  and  determine  what 
changes  must  be  made.  The  following  is  an  outline  of  the  review  process 
for  procedural  tasks 

a.  Review  Task  Documentation.  Information  about  procedural  tasks 
can  be  obtained  from  a  variety  of  sources.  Review  the  documents  in 
order  to  construct  an  accurate  list  of  tasks  trained  in  the  current 
course.  For  various  reasons,  the  documents  may  not  agree  with  each 
other.  Note  each  discrepancy  for  discussion  in  the  following  activity. 
The  following  documents  should  be  used  to  determine  the  list  of 
procedural  tasks: 

(1)  The  Current  POI.  This  document  provides  the  most  basic 
reference  on  what  tasks  are  being  trained  in  the  current  course.  It 
lists  all  the  tasks  within  clusters  as  well  as  other  information  about 
the  conduct  and  content  of  the  course. 

(2)  The  Master  Task  List.  This  task  list  for  the  Career 
Management  Field  (CMF)  presents  an  inventory  of  all  tasks  in  the  CMF 
that  subsumes  the  Military  Occupation  Specialty  (MOS)  in  question.  The 
Master  Task  List  also  indicates  the  suggested  skill  level  (1-5)  for  each 
task  as  well  as  the  appropriate  site  for  training  (institution,  exten¬ 
sion,  or  not  recommended  for  training). 

(3)  The  Soldier's  Manual.  This  manual  provides  a  Training 
Plan  at  the  beginning  of  the  manual  which  lists  each  task  in  the  MOS 
along  with  the  appropriate  level  for  training  the  task  (entry-level, 
primary  NCO,  basic  NCO,  etc.). 

(4)  The  Training  Schedule.  The  schedule  for  the  current 
course  lists  the  tasks  in  the  sequence  in  which  they  are  currently  being 
trained. 


(5)  Information  About  New  Equipment.  If  the  primary  reason 
for  revising  the  course  is  to  introduce  new  equipment,  existing  infor¬ 
mation  about  the  equipment  (e.g.,  draft  operator's  manual)  must  be 
obtained.  Such  documents  must  be  carefully  reviewed  to  identify  any  new 
tasks  that  need  to  be  added  to  the  inventory  or  any  existing  tasks  that 
need  to  be  modified. 


b.  Interview  Personnel. 

(1)  Solicit  the  opinions  of  personnel  who  are  knowledgeable 
about  the  present  course  concerning  the  revised  tasks  inventory  and  the 
overall  course  goal.  First,  prepare  a  tentative  list  of  tasks  identi¬ 
fied  from  the  documentation.  Present  this  list  to  a  small  group  of 
experts  (less  than  eight)  for  their  comments.  Have  the  group  review  the 
list  on  a  task-by-task  basis  and  have  group  members  respond  as  to 
whether  they  think  that  the  task  should  be  included  in  the  revised 
course,  dropped  from  the  revised  course,  or  modified  from  its  present 
form.  In  addition,  have  the  interviewees  read  and  comment  on  the  course 
objective  as  it  presently  stands.  It  is  probably  a  good  idea  to  tape 
record  the  session  to  prevent  your  having  to  take  copious  notes. 

(2)  Below  are  some  suggested  groups  who  can  provide  useful 
input  to  the  development  of  a  revised  task  inventory.  In  addition  to 
the  task-by-task  review  discussed  above,  the  group  members  should  also 
address  the  following  points: 

(a)  Training  Development  Personnel.  Ask  these  personnel 
about  the  discrepancies  in  task  documents  identified  earlier. 

(b)  Instructors  for  the  Present  Course.  Ask  instructors 
to  identify  tasks  that  are  presently  difficult  to  train  and  to  learn. 

(c)  Course  Graduates.  Ask  course  graduates  to  identify 
tasks  that  are  difficult  to  perform  on  the  job. 

(d)  Course  Evaluators.  Obtain  any  information  internal 
evaluation  personnel  may  have  about  the  performance  of  students  in 
formal  testing  situations.  External  evaluators  should  be  asked  about 
any  significant  feedback  they  might  have  received  about  course  graduates 
from  the  field. 

(e)  Transition  Trainers.  If  the  primary  reason  for 
revising  the  course  is  to  introduce  new  equipment,  any  personnel  who  are 
assigned  to  transition  train  present  job  incumbents  on  the  new  equipment 
provide  an  invaluable  source  of  information.  They  should  specifically 
address  the  impact  of  the  new  equipment  on  the  task  Inventory. 

c.  Prepare  Draft  of  the  Procedural  Task  List.  On  the  basis  of 
the  existing  task  list  and  suggested  changes,  prepare  a  draft  of  the 
revised  task  list.  Be  sure  to  indicate  which  tasks  have  been  added  to 
or  deleted  from  the  list  and  indicate  which  tasks  have  been  modified. 
Also  include  a  clear  and  concise  rationale  for  each  proposed  change  to 
the  existing  list.  As  a  general  rule,  it  is  better  to  include  than 


exclude  tasks  at  this  point.  The  list  will  be  pared  down  later  in  the 
course  revision  process. 


4-3.  Perform  Limited  Reanalysis  of  Job.  At  this  point  in  the  task  list 
revision,  you  should  look  beyond  traditional  military  sources  to  assure 
the  completeness  of  the  task  list.  Task  documents  may  be  too  narrowly 
focused  on  procedural  tasks.  Nonprocedural  tasks  (e.g.,  decision  making 
or  others  that  are  primarily  mental  in  nature)  may  have  been  ignored  in 
the  original  job  analysis.  If  this  is  not  the  case,  you  may  skip 
Section  4-3  and  go  to  Section  4-4.  However,  if  you  have  reason  to 
believe  that  traditional  job  analyses  have  ignored  tasks  that  are 
important  to  job  performance,  you  should  consider  performing  a  limited 
reanalysis  of  the  job.  It  should  be  noted  that  the  sorts  of  job 
analyses  described  in  IPISD  and  other  documents  are  expensive  and 
time-consuming  procedures.  What  is  described  here  Is  meant  to  be  a 
limited  reanalysis  which  should  take  a  single  analyst  (e.g.,  the  course 
reviser  himself)  less  than  two  weeks  to  complete. 

a.  Identify/Define  the  Nonprocedural  Activities. 

(1)  The  reanalysis  of  the  job  should  be  limited  in  the  sense 
that  it  focuses  on  just  a  few  types  of  important  nonprocedural  activi¬ 
ties  that  have  been  identified  as  important  to  job  performance.  The 
type  of  nonprocedural  activity  should  be  sufficiently  defined  so  that  an 
analyst  clearly  understands  the  activity  in  question. 

(2)  Using  the  19K  BNCOC  example,  it  was  felt  that  one  of  the 
nonprocedural  activities  overlooked  by  traditional  analyses  of  the  tank 
commander's  job  was  decision  making.  In  the  context  of  commanding  a 
tank,  decision  making  was  defined  as  behaviors  where  the  tank  commander 
is  confronted  with  a  situation  in  which  alternative  responses  are 
possible,  and  he  must  decide  which  response  is  most  appropriate. 

b .  Identify  Functional  Areas. 

(1)  In  order  to  provide  structure  to  the  analysis,  you  should 
identify  between  5  and  10  basic  functions  of  the  job.  Possible  func¬ 
tional  areas  are  listed  as  task  categories  in  the  Soldier's  Manual. 

Many  functions  are  common  to  all  combat  arms  MOS. 

(2)  For  the  19K  BNCOC  example,  the  following  functional  areas 
were  identified  for  decision  making  tasks: 

(a)  Movement , 

(b)  Detection/Identification, 

(c)  Gunnery, 

(d)  Sustainment, 

(e)  Communications, 


(f)  Training,  and 

(g)  Personnel. 


c.  Identify  Activities  within  Functional  Areas. 

(1)  Within  each  of  the  functional  categories,  you  should 
identify  the  mental  activities  that  relate  to  the  functions.  Strive  for 
a  fairly  complete  listing  of  activities  without  regard  to  exact  military 
or  psychological  terminology  or  to  the  contents  of  the  present  training 
program.  SMEs  may  have  trouble  identifying  nonprocedural  activities 
because  of  their  dissimilarity  to  traditional  procedural  tasks.  When 
finished,  have  the  list  of  nonprocedural  tasks  reviewed  by  other  SMEs 
for  completeness. 

(2)  The  method  used  to  identify  the  decision  making  tasks 
relevant  to  19K  BNCOC  was  to  chronologically  review  one  or  two  repre¬ 
sentative  combat  scenarios  and  list  the  mental  activities  that  occur. 

The  decision  making  tasks  were  then  rearranged  under  functional 
headings.  As  as  example,  the  analysis  revealed  the  following  tank 
commander  decision  making  tasks  that  fall  under  the  functional  heading 
of  Gunnery: 


(a)  Decide  Whether  or  Not  to  Fire  at  Target (s) , 

(b)  Decide  Sequence  in  Which  to  Engage  Multiple  Targets, 

(c)  Decide  When  to  Fire  at  Target (s), 

(d)  Decide  What  Weapon  to  Fire, 

(e)  Choose  Appropriate  Main  Gun  Ammunition,  and 

(f)  Decide  When  to  Stop  Firing. 

d .  Relate  Nonprocedural  to  Procedural  Tasks. 

(1)  Many  of  the  nonprocedural  tasks  coincide  with  or  occur 
during  the  execution  of  one  or  more  of  the  procedural  tasks.  To  system¬ 
atically  examine  the  relationship  between  procedural  and  nonprocedural 
tasks,  you  should  construct  a  crosswalk  of  the  two  types  of  tasks.  The 
crosswalk  is  a  matrix  which  lists  all  the  procedural  tasks  on  one  dimen¬ 
sion  and  all  nonprocedural  tasks  on  the  other.  An  analyst  should  then 
indicate  which  of  the  nonprocedural  activities  occur  during  the  execu¬ 
tion  of  each  procedural  task.  The  purpose  of  the  crosswalk  is  to 
identify  where  a  nonprocedural  activity  might  be  trained  within  the 
context  of  procedural  instruction.  If  an  important  nonprocedural 
activity  is  not  related  to  any  procedural  task,  you  should  add  the  task 
to  the  inventory.  An  alternative  course  of  action  is  to  develop  a  more 
standard  procedural  task  which  incorporates  the  nonprocedural  activity 
in  question. 


(2)  Such  matrices  were  constructed  for  19K  BNCOC.  The 
detailed  findings  were  presented  by  listing  each  nonprocedural  task  and 
the  procedural  tasks  to  which  they  apply.  Figure  4-1  presents  an 
excerpt  of  the  findings  for  only  the  decision-making  that  pertain  to 
target  detection  and  identification.  The  numbered  items  are  the  non¬ 
procedural  tasks,  whereas  the  subordinate  items  are  the  procedural  tasks 
to  which  they  are  related. 


DETECTION/ IDENTIFICATION 

1.  Decide  Whether  or  Not  to  Override  Designated  Search  Area 

a.  Conduct  Target  Acquisition  (Tactics) 

b.  Direct  Evasion  of  an  Enemy  Anti-Tank  Guided  Missile  (Tactics) 

2.  Decide  WTiere  TC  and  Loader  Will  Search  for  Targets 

a.  Conduct  Target  Acquisition  (Tactics) 

b.  Direct  Evasion  of  an  Enemy  Anti-Tank  Guided  Missile  (Tactics) 

3.  Decide  Which  Mode  of  Observation  Will  Be  Used  to  Search  for 
Targets 

a.  Conduct  Target  Acquisition  (Tactics) 

b.  Direct  Evasion  of  an  Enemy  Anti-Tank  Guided  Missile  (Tactics) 


Figure  4-1.  Excerpt  of  Crosswalk  Between  Nonprocedural 
and  Procedural  Tasks 


4-4,  Finalize  the  Task  Inventory. 

a.  Review  Revised  Task  Inventory.  After  reconciling  procedural 
and  nonprocedural  tasks,  you  should  reexamine  the  task  list  as  a  whole 
to  determine  whether  the  tasks  are  in  keeping  with  the  course  goal.  It 
is  certainly  possible  that,  after  the  reanalysis  of  the  training 
requirements,  there  may  be  some  modification  of  the  course  objective. 

b.  Submit  Inventory  for  Approval.  After  reviewing  the  tasks, 
submit  the  list  to  training  management  for  their  approval.  Appended  to 
the  list  should  be  reasons  for  any  additions,  deletions,  or  modifica¬ 
tions  of  the  current  Inventory.  Also  submit  any  proposed  changes  to  the 
course  objective  along  with  the  inventory.  You  should  try  to  get 
management  to  do  more  than  "sign  off"  on  the  list;  you  should  actively 
solicit  their  suggestions  for  changes  so  that  they  have  input  in  deter¬ 
mining  course  contents.  They  may  even  wish  other  SMEs  to  review  your 
work.  Consolidate  all  the  suggested  changes  along  with  the  rationale 
for  the  changes  in  a  memorandum  for  record. 


CHAPTER  5 


SELECT  TASKS  FOR  TRAINING 


5-1.  Introduction.  In  the  previous  phase  (Revise  Task  Inventory),  the 
emphasis  was  on  completeness.  In  all  likelihood,  your  inventory  of 
tasks  probably  contains  too  many  tasks  to  be  trained  in  any  one  course. 
The  solution  is  to  select  only  the  most  important  tasks  for  training. 

In  the  current  procedure,  tasks  are  selected  for  training  by  a  two-stage 
process.  In  the  first  stage,  tasks  are  rated  in  terms  of  their  criti¬ 
cality  by  subject  matter  exrerts  (SMEs) .  In  the  second  stage,  training 
experts  use  task  criticality  3nd  other  information  to  select  tasks  which 
should  be  in  the  course. 

5-2.  Obtain  Task  Criticality  Ratings.  Obtain  task  criticality  data  by 
surveying  an  appropriate  sample  of  SMEs. 

a.  Construct  Task  Survey.  Individual  task  survey  forms  are  used 
to  obtain  criticality  ratings  from  SMEs.  According  to  IPISD  procedures 
(i.e.,  TRADOC  Pamphlet  350-30),  there  are  eight  different  dimensions  of 
task  criticality  on  which  tasks  may  be  rated.  SMEs  have  found  the 
requirement  to  rate  tasks  on  multiple  criteria  tedious  and  not  relevant 
to  training  development.  In  contrast,  the  recommended  method  here  is  to 
have  SMEs  rate  tasks  on  a  single  dimension  of  criticality  that  is  more 
directly  related  to  job  incumbents.  The  dimension  being  rated  is  how 
important  each  task  is  for  the  job  incumbent  to  know.  To  construct  an 
appropriate  task  survey,  take  the  following  steps: 

(1)  Develop  Rating  Scale.  The  rating  scale  should  use  termi¬ 
nology  with  which  military  personnel  are  familiar.  The  following 
four-point  rating  scale  is  recommended: 


(a) 

must  know:  4 

points. 

(b) 

should  know: 

3  points. 

(c) 

nice  to  know: 

2  points,  and 

(d) 

no  need  to  know:  1  point. 

(2)  Develop  Survey  Format.  The  survey  should  be  constructed 
such  that  it  Is  easy  for  the  SME  to  understand  and  easy  for  you  to 
score.  List  tasks  separately  so  that  SMEs  can  indicate  their  rating  for 
each  task.  In  the  example  page  from  the  19K  BNCOC  survey  (Figure  5-1), 
the  SME  indicates  his  rating  of  each  task  by  circling  the  appropriate 
number  beside  each  item.  Rating  Instructions  should  be  repeated  at  the 
top  of  every  page  of  the  survey  to  keep  SMEs  mindful  of  the  rating 
scale. 


INSTRUCTIONS :  The  following  is  a  list  of  tasks  which  could  be 


performed  by  the  tank  commander.  How  important  is  it  for  the  tank 

commander  to  know  how  to  perform  these  tasks?  Circle  the  number 

corresponding  to  your  rating  using  the  following  scale: 

: _ 1 _  :  _ 2 _  :  _ 3 _  :  _ 4 _ 

No  Need  to  Know  Nice  to  Know  Should  Know  Must  Know 


Encode/Decode  Messages  Using  KTC  600D  Tactical 

Operations  123 


Conduct  Training 


1  2  3 


Read/Report  Radiation  Dosages 


1  2  3 


Use  Marginal  Information  on  a  Map 

Prepare  Commander's  Weapon  Station  (CWS)  for 
Operation  on  an  Ml  Tank 

Determine  a  Location  on  the  Ground  by  Terrain 
Association 


1  2  3 

1  2  3 

1  2  3 


Engage  Targets  with  the  Caliber  .50  M2  HB 
Machinegun  on  an  Ml  Tank 

Determine  Azimuth  Using  a  Protractor  and  Compute 
a  Back  Azimuth 

Conduct  a  Map  Reconnaissance 
Call  for  and  Adjust  Indirect  Fire 
Employ  a  Three-Man  Crew 

Boresight  a  Caliber  .50  M2  HB  Machinegun  on 
an  Ml  Tank 


Supervise  Before  Operations  Checks  and 

Services  on  an  Ml  Tank  123 


Prepare/Submit  NBC-1  Report 


2  3 


4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 


(3)  Develop  Instructions.  A  cover  sheet  (see  Figure  5-2) 
should  present  a  brief  explanation  of  purpose  and  importance  of  the 
survey  followed  by  the  rating  instructions.  In  the  present  example, 
general  instructions  are  presented  on  the  cover  sheet  (Figure  5-2)  with 
more  specific  instructions  on  the  first  page  of  the  survey  (Figure  5-1). 
Also,  note  that  additional  information  concerning  the  SME's  level  of 
experience  is  obtained  by  items  on  the  cover  sheet. 


TASK  CRITICALITY  SURVEY 


b.  Obtain  Appropriate  SMEs.  In  order  to  obtain  valid  ratings  of 
task  criticality,  the  SMEs  must  have  had  first-hand  experience  in  per¬ 
forming  the  tasks.  In  other  words,  they  should  have  been  incumbents  in 
the  job  in  question  long  enough  to  have  had  some  experience  with  all  the 
tasks  in  the  inventory.  The  sample  of  SMEs  should  be  neither  too  small 
so  that  task  ratings  are  unreliable,  nor  too  large  so  that  personnel 
costs  associated  with  the  survey  exceed  the  relatively  small  gains  in 
reliability.  The  sample  size  should  probably  be  no  smaller  than  10  and 
no  larger  than  30. 

c.  Analyze  Results.  Assemble  the  completed  surveys  and  take  the 
following  measures  to  analyze  the  results: 

(1)  Compile  Data.  Prepare  a  data  sheet  with  individual  tasks 
as  rows  and  SME  responses  as  columns.  Figure  5-3  presents  an  example 
using  a  few  tasks  from  19K  BNCOC.  In  this  hypothetical  situation,  three 
SMEs  (a,  b,  and  c)  have  rated  three  tasks.  Normally,  more  SMEs  would  be 
used  to  rate  a  larger  selection  of  tasks.  The  numbers  were  kept  small 
to  simplify  the  arithmetic. 


Tasks 

Raters 
a  b  c 

Average 

Rating 

Relative 

Rank 

Conduct  Training 

3  2  2 

2.33 

3 

Call  for  and  Adjust  Indirect  Fire 

4  4  4 

4.00 

1 

Employ  a  Three -Man  Crew 

4  3  3 

3.33 

2 

Figure  5-3.  Example  Data  Sheet  for  Compiling  SME  Ratings 


(2)  Compute  Average  Ratings.  For  each  task,  compute  the 
average  rating.  Do  this  by  adding  all  of  the  ratings  for  each  task  and 
dividing  by  the  number  or  raters  (SMEs).  Each  calculation  should  be 
carried  out  to  three  decimal  places  and  rounded  to  two.  The  average 
ratings  for  the  example  tasks  are  shown  in  the  next  to  last  column  of 
Figure  5-3. 

(3)  Rank  the  Tasks.  Rank  order  the  tasks  from  highest  to 
lowest  average  rating.  The  ranking  allows  you  to  see  relative  position 
of  tasks  in  terms  of  criticality.  The  task  ranking  should  generally 
conform  to  your  common  sense  notions  of  the  relative  importance  of  the 
tasks.  The  rank  order  of  the  example  tasks  is  indicated  in  the  last 
column  of  Figure  5-3.  The  number  one  (1)  indicates  that  the  task 
entitled  "Call  for  and  Adjust  Indirect  Fire"  received  the  highest 
rating;  the  number  two  (2),  the  second  highest  rating;  and  so  on. 


1.  Conduct  Delphi  Negotiation  to  Select  Tasks.  The  SME  ratings 
provide  an  important,  but  by  no  means,  exclusive  criterion  for  task 
selection.  The  task  selection  process  should  also  be  based  on  other 
factors  such  as  training  management  considerations  (e.g.,  training  time, 
costs,  and  other  resource  issues),  instructional  considerations  (e.g., 
task  difficulty  or  instructional  sequencing),  and  military  considera¬ 
tions  (e.g,,  the  relationship  of  a  task  to  the  unit's  mission).  A  panel 
of  training  experts  should  be  convened  to  consider  these  issues  and  the 
SME  ratings  in  order  to  select  and  prioritize  tasks  to  be  trained  in  the 
revised  course. 

a.  Determine  Partitioning  Rules.  After  inspecting  the  task 
inventory,  training  management  may  have  given  you  guidance  as  to  what 
percentage  of  the  tasks  should  be  chosen  for  training.  If  their 
guidance  is  less  explicit,  devise  your  own  partitioning  rules. 
Partitioning  rules  refer  to  how  the  tasks  will  be  divided  to  indicate 
which  will  be  trained.  The  simplest  rule  is  to  have  two  categories: 
those  tasks  that  will  be  trained  and  those  that  will  not  be  trained. 

The  recommended  rule  is  to  divide  the  task  inventory  into  three  parts. 
The  procedure  is  described  below: 

(1)  According  to  IPISD  (TRADOC  Pamphlet  350-30,  Vol.  1, 
p.  143),  the  task  inventory  should  be  divided  into  three  parts:  the 
most  important  60-65%  of  the  tasks  (referred  to  as  "must  train”  tasks) , 
the  least  important  15-20%  of  the  tasks  ("don’t  train"  tasks),  and  the 
remaining  15-25%  of  the  tasks  that  should  be  trained  if  time  allows 
("should  train"  tasks).  The  "should  train"  tasks  are  then  prioritized 
from  most  to  least  important.  This  middle  category  allows  the  course 
contents  to  be  systematically  varied  as  time  and  resource  constraints 
change. 


(2)  Do  not  be  limited  to  the  proportions  suggested  in  IPISD 
if  your  situation  demands  otherwise.  For  instance,  if  you  think  that 
there  will  be  enough  time  to  train  most  of  the  tasks  on  the  inventory, 
you  may  wish  to  make  the  least  important  category  smaller.  Or,  if  you 
need  to  provide  training  management  more  latitude  in  modifying  course 
length,  you  may  want  to  make  the  middle  category  larger.  Take  time  to 
consider  your  situation  and  devise  a  reasonable  method  for  partitioning 
the  task  inventory. 

b.  Obtain  Task  Selection  Panel.  As  stated  above,  a  wide  range  of 
factors  should  be  considered  in  the  task  selection  process.  Conse¬ 
quently,  the  task  selection  panel  should  represent  a  diversity  of 
backgrounds  related  to  training.  Appropriate  experts  include  instruc¬ 
tors  in  the  present  course,  training  analysts,  training  developers,  etc. 
Training  management  may  wish  to  participate  or  to  assign  a  representa¬ 
tive.  To  keep  the  negotiation  manageable,  no  more  than  eight  people 
should  serve  on  the  task  selection  panel.  Furthermore,  to  lend  greater 
objectivity  to  the  process,  you  (the  course  developer/reviser)  should 
not  serve  as  a  panel  member  and  should  act  only  as  moderator  of  the  task 
selection  process. 
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c.  Conduct  Delphi  Negotiation.  Some  members  of  the  panel  may  be 
inhibited  by  face-to-face  meetings  because  of  differences  in  rank  or 
because  of  a  reluctance  to  speak  in  front  of  groups.  As  a  consequence, 
an  important  source  of  expertise  may  be  effectively  lost  to  the  group. 

To  overcome  this  problem,  the  task  selection  panel  will  start  the 
negotiation  using  an  anonymous  group  problem  solving  method  called  the 
Delphi  technique.  The  following  negotiation  procedures  should  be  used 
to  select  tasks  for  training: 

(1)  Instruct  Panelists.  Give  the  panelists  the  results  of 
the  SME  ratings  to  use  in  making  their  task  selections.  Explain  that 
there  will  be  several  rounds  of  negotiation  wherein  panelists  interact 
anonymously  to  achieve  group  consensus.  For  each  round  of  negotiation, 
instruct  panelists  to  divide  the  tasks  according  to  the  partitioning 
rules.  That  is,  panelists  should  place  so  many  tasks  (exact  number 
depends  on  the  partitioning  rules)  in  the  "must  train"  category,  so  many 
tasks  in  the  "don't  train"  category,  and  the  rest  in  the  "should  train" 
category.  Panelists  should  work  independently  and  at  their  own  rate. 

The  moderator  should  try  not  to  reveal  the  identify  of  the  panelists  to 
one  another  during  the  negotiation  process. 

(2)  Feed  Back  Results.  After  each  round  of  negotiation,  you 
(the  moderator)  should  prepare  a  report  on  the  results.  An  example 
report  is  shown  in  Figure  5-4.  The  report  should  have  the  following 
features : 


(a)  Response  Frequencies.  Count  how  many  panelists 
responded  to  either  "must  train,"  "should  train,"  or  "don't  train"  to 
each  task.  The  frequency  of  responses  to  each  task  should  sum  to  the 
total  number  of  panelists.  In  the  example  report  (Figure  5-4),  it  can 
be  seen  that  a  total  of  five  panelists  have  divided  the  three  tasks  that 
were  previously  rated  by  SMEs  (Figure  5-3) . 


_ Frequencies _ 

Must  Should  Don't 

Task _ Train  Train  Train  Comments 


Conduct  Training 

Call  for  and  Adjust 
Indirect  Fire 

Employ  a  Three-Man 
Crew 


0  3  2 

2  3  0 

©  3  1 


Combat  will  probably 
require  employment 
of  three-man  crews; 
TCs  are  currently 
not  trained  on  the 
task. 


Figure  5-4.  Example  Feedback  Report 


(b)  Minority  Opinions.  Indicate  whenever  a  panelist  is 
in  the  minority  of  opinion  by  circling  the  category  in  which  his  or  her 
response  fell.  A  minority  response  is  where  a  panelist  gives  the  least 
frequently  chosen  response  category  for  a  given  task.  In  the  example 
individual  feedback  report  (Figure  5-4),  the  "must  train"  alternative  is 
circled  indicating  that  the  hypothetical  panelist  was  the  only  one  who 
classified  the  task  entitled  "Employ  a  Three-Man  Crew"  as  a  "must  train" 
item.  Minority  responses  can  be  seen  for  the  other  two  tasks  as  well  as 
for  the  one  panelist  who  classified  the  three-man  crew  task  as  a  "don't 
train"  item.  The  fact  that  other  categories  are  not  circled  for  the 
other  two  tasks  indicates  that  the  hypothetical  panelist  responded  with 
the  majority  in  those  cases.  Whenever  a  minority  response  is  so  identi¬ 
fied,  the  panelist  is  instructed  to  either  change  his  or  her  response  on 
the  next  round  of  negotiation  or  to  defend  his  or  her  response  in 
writing. 

(c)  Comments .  If  a  panelist  refuses  to  change  a 
minority  opinion  and  submits  a  written  reason  for  not  changing  his 
response,  his  reasoning  is  briefly  summarized  in  the  comments  column  of 
the  feedback  report  and  subsequently  circulated  to  the  other  panelists 
along  with  the  results  of  the  next  round  of  negotiation. 

(3)  Restrict  Responses.  In  order  to  maintain  the  proportions 
of  tasks  within  categories,  any  change  in  the  response  to  a  task  in  a 
particular  category  must  be  compensated  by  a  change  in  classification  of 
a  task  in  another  category.  In  order  to  prevent  wholesale  changes  in 
responses  and  to  promote  group  consensus,  panelists  should  not  be 
allowed  to  change  their  response  to  tasks  that  have  been  "frozen"  into  a 
particular  category.  Frozen  tasks  are  those  whose  category  was  unami- 
mously  agreed  upon  in  a  previous  round.  This  restriction  in  responding 
has  the  effect  of  eliminating  frozen  tasks  from  consideration  and 
focusing  the  group's  attention  on  tasks  on  which  they  disagree. 

d.  Resolve  Remaining  Differences.  Over  the  course  of  3-7  rounds 
of  negotiation  (depending  on  the  size  of  the  task  inventory),  the  task 
selection  panel  will  come  to  a  relatively  quick  consensus  for  most 
(80-90 Z)  of  the  tasks.  There  probably  will  be,  however,  some  panel 
members  who  refuse  to  change  their  mind  about  particular  tasks  even  if 
they  are  in  the  minority.  Instead  of  continuing  the  Delphi  indefi¬ 
nitely,  there  are  two  measures  you  can  take  to  resolve  the  remaining 
differences  between  panel  members: 


(1)  Convene  a  Face-to-Face  Meeting.  The  first  measure  is  to 
convene  a  face-to-face  meeting  of  panelists.  The  purpose  of  this 
meeting  is  for  the  panelists  holding  minority  opinions  to  more  fully 
explain  their  reasons  for  their  responses  and  for  others  to  react  to 
those  reasons.  As  moderator  of  this  meeting,  you  should  insure  that 
discussion  only  concern  the  tasks  about  which  the  panelists  disagree. 
Hopefully,  all  differences  can  be  resolved  in  this  meeting.  If  the 
panel  does  not  resolve  all  disagreements,  you  need  to  take  the  following 
measure. 


(2)  Make  Decision.  For  the  remaining  tasks,  you  should 
consider  all  the  arguments  presented  and  make  an  appropriate  decision. 
Your  judgment  should  be  based  on  the  content  of  panelists'  arguments 
and  not  on  how  well  the  arguments  are  presented.  If  plausible  arguments 
are  presented  on  both  sides  of  an  issue,  you  should  go  with  the  majority 
opinion.  Organize  the  task  inventory  into  a  proposed  "must,"  "should," 
and  "don't"  train  categories  and  submit  to  the  panel  for  their  approval 
and  comments. 

e.  Prioritize  Tasks  in  the  "Should  Train"  Category.  There  are  at 
least  three  methods  for  prioritizing  tasks  in  the  "should  train"  cate¬ 
gory  from  the  most  to  the  least  important: 

(1)  Panelists  Rank  Tasks.  If  task  selection  comes  to  a 
fairly  quick  conclusion,  you  can  ask  the  panelists  to  prioritize  the 
tasks  from  most  to  least  important  for  training  purposes.  Because 
ranking  requires  a  great  many  comparative  judgments  between  tasks,  this 
procedure  should  only  be  used  when  there  are  not  many  tasks  (less  than 
15)  in  the  "should  train"  category.  Also,  panelists  are  not  likely  to 
unanimously  agree  on  task  rankings.  You  either  have  to  devise  a  method 
for  combining  group  opinion  (e.g.,  taking  the  average  rank  for  each 
task)  or  engage  in  another  negotiation  to  obtain  a  group  consensus. 

(2)  Use  SME  Ratings.  A  much  easier  method  of  prioritizing 
tasks  is  to  rank  tasks  on  the  basis  of  average  ratings  assigned  tasks  by 
the  SMEs  in  the  first  phase  of  task  selection.  As  argued  earlier, 
however,  the  SMEs  may  not  be  sensitive  to  training  considerations. 

(3)  Use  Results  from  Negotiation.  The  recommended  method  of 
prioritizing  tasks  is  to  use  the  responses  of  the  task  selection  panel. 
The  panelists  have,  in  effect,  repeatedly  rated  the  tasks  throughout  the 
task  selection  process.  Use  the  following  procedures  to  convert  the 
panel's  responses  to  relative  priorities  among  the  "should  train"  Items: 

(a)  Quantify  Responses.  To  quantify  their  responses, 
assign  the  "don't  train,"  "should  train,"  and  "must  train"  responses 
numerical  values  of  one,  two,  and  three  points  respectively.  For  each 
round,  multiply  the  point  values  of  each  response  by  the  number  of 
panelists  giving  that  particular  response.  The  example  in  Figure  5-5  is 
taken  from  the  results  reported  in  the  previous  figure  (5-4).  On  this 
single  round  of  negotiation,  tasks  are  then  prioritized  as  shown  in  the 
last  column  of  the  Figure. 


(b)  Combine  Results  from  All  Rounds.  To  combine  the 
results  from  all  rounds,  you  can  simply  sum  the  points  across  rounds. 
However,  this  procedure  weights  the  results  from  each  round  equally. 

The  later  rounds  are  more  important  because  they  more  fairly  represent 
the  group's  convergence  of  opinion.  An  alternative  procedure  is  to  rank 
the  tasks  on  the  basis  of  the  last  round.  Ties  are  to  be  expected  since 
most  panelists  respond  "should  train"  bv  the  end  of  negotiations.  Ties 
can  be  resolved  by  the  number  of  points  the  tasks  received  in  the  next 
previous  round,  and  so  forth  until  all  ties  are  resolved. 


5-4.  Prepare  Report  on  Results.  Prepare  a  report  on  the  task  selection 
process  for  training  management.  The  report  should  contain  both  the 
method  and  the  results  of  this  phase  in  course  revision.  The  report 
should  be  labeled  for  information  purposes  only  since  comments  or 
approval  are  not  required  at  this  point  in  the  revision  process. 


Conduct  Training 

Call  for  and  Adjust 
Indirect  Fire 

Employ  a  Three -Man  Crew 


Response  Category 

Must  Should  Don't  Total  Relative 
Train  Train  Train  Points  Priority 

(0x3)  +  (3x2)  +  (2x1)  =8  3 


(2x3)  +  (3x2)  +  (0x1)  =  12  1 

(1x3)  +  (3x2)  +  (lxl)  -  10  2 


Figure  5-5.  Recommended  Method  of  Prioritization 
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CHAPTER  6 


REVISE/DEVELOP  TRAINING  OBJECTIVES 


6-1.  Introduction.  Training  objectives  must  now  be  developed  for  each 
task  selected  for  training. 

a.  Training  objectives  are  sometimes  referred  to  as  "terminal" 
objectives  because  they  describe  the  final  behaviors  that  students  are 
supposed  to  acquire  as  a  result  of  instruction.  These  terminal  objec¬ 
tives  represent  the  second  level  in  a  hierarchy  of  course  objectives. 
That  is,  training  objectives  are  subordinate  to  the  overall  course 
objective.  Training  objectives  may  also  be  broken  down  into  subcom¬ 
ponents  called  "enabling"  objectives.  These  enabling  objectives  are 
identifiable  subobjectives  that  must  be  attained  In  order  to  attain  the 
training  objective.  The  individual  performance  elements  are  the  lowest 
level  of  analysis  and  usually  represent  an  observable  task  action  or 
step. 


b.  Training  objectives  have  already  been  developed  for  the  tasks 
that  are  in  the  current  POI  (referred  to  as  "old"  tasks).  The  training 
objectives  for  each  of  the  old  tasks  should  be  systematically  examined 
and  modified  as  required.  For  tasks  that  are  not  already  In  the  current 
POI  or  those  that  are  substantially  modified  (i.e.,  the  "new"  tasks), 
training  objectives  must  be  developed  anew.  These  two  processes  are 
described  in  separate  sections  below. 

6-2.  Revise  Objectives  of  Old  Tasks.  Training  objectives  are  derived 
from  detailed  analyses  of  tasks  into  enabling  objectives  and  performance 
elements.  Unfortunately,  it  is  sometimes  difficult  to  obtain  the 
original  documentation  for  these  analyses.  However,  the  final  product 
of  task  analysis  (i.e.,  the  training  objectives  themselves)  can  be  found 
in  multiple  sources. 

a.  Obtain  Statements  of  Objectives.  There  are  several  sources 
from  which  you  can  obtain  statements  of  training  objectives: 

(1)  Program  of  Instruction.  A  statement  of  each  training 
objective  in  the  existing  course  can  be  obtained  from  the  POI. 

(2)  Lesson  Plans.  Training  objectives  can  also  be  found  at 
the  beginning  of  the  appropriate  lesson  plan  for  the  existing  course. 

(3)  Soldier’s  Manuals.  All  procedural  tasks  should  be 
described  in  either  the  Soldier's  Manual  for  the  MOS  or  the  Soldier's 
Manual  of  Common  Tasks. 

b.  Identify  Structural  Components.  Training  objectives  consist 
of  three  structural  components: 

(1)  Actions :  Behavior(s)  required  to  execute  a  particular 
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task. 


(2)  Conditions :  Important  aspects  of  the  environment  that 
are  present  during  task  performance. 

(3)  Standards :  Criteria  of  performance  that  must  be  attained 
in  order  to  successfully  execute  the  task. 

c.  Review  for  Consistency.  The  actions,  conditions,  and 
standards  as  stated  in  the  POI  and  in  the  Lesson  Plans  should  be  in 
total  agreement  for  each  training  objective.  Note  wherever  they  in  fact 
disagree.  These  discrepancies  must  be  resolved  before  proceeding  with 
development  of  the  revised  course.  Many  of  the  discrepancies  are  easily 
resolved.  Others  may  require  you  to  go  to  some  authoritative  source 
such  as  the  Soldier's  Manual  or  SMEs.  Be  cautioned,  especially  for 
common  tasks,  that  the  Soldier's  Manual  task  statement  may  be 
significantly  different  from  the  statement  for  the  corresponding  task 
which  has  been  tailored  to  a  particular  course.  To  take  an  example  from 
19K  BNCOC,  the  task  entitled  "Navigate  from  One  Point  on  the  Ground  to 
Another"  requires  the  soldier  to  navigate  over  a  distance  of  5,000 
meters  whereas  the  Soldier's  Manual  task  requires  only  a  distance  of 
3,000  meters.  The  difference  is  that  the  former  (armor)  task  assumes 
that  the  soldier  is  in  a  tank  and  the  latter  task  (evidently  developed 
for  infantry)  assumes  the  soldier  does  not  have  a  vehicle.  Such 
discrepancies  between  tasks  as  stated  in  the  Soldier's  Manual  and  those 
in  the  existing  course  are  perfectly  reasonable  and,  therefore, 
tolerable . 

d.  Review  for  Adequacy.  Once  you  have  obtained  consistent 
statements  of  training  objectives,  you  should  evaluate  their  adequacy 
using  the  following  criteria  (taken  from  TRADOC  Pamphlet  350-30,  Volume 
II,  p.  8-9): 

(1)  Objectives  must  be  a  statement  of  student  behaviors 
(actions),  such  as  the  creation  of  a  product  or  some  other  overt  act, 
which  can  be  accepted  as  evidence  that  the  intended  outcome  has 
occurred. 

(2)  The  behavior  must  describe  specifically  all  steps  and 
outcomes  that  will  demonstrate  that  learning  has  occurred. 

(3)  The  student  behavior  called  for  must  be  capable  of 
observation  and  evaluation  within  the  learning  and  testing  environments. 

(4)  The  objective  must  be  stated  in  learner  rather  than 
teacher  terms,  i.e.,  actions  which  the  student  will  perform  rather  than 
what  the  teacher  will  say  or  do. 

(5)  There  must  be  a  standard  against  which  the  student 
behavior  will  be  measured.  It  must  be  fully  specified. 

(6)  The  statement  of  the  conditions  under  which  the  student 
will  occur  must  be  fully  specified. 
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>.  Revise  Objective  as  Appropriate.  Any  changes  identified  as 
appropriate  in  your  review  should  be  incorporated  into  the  training 
objectives. 

6-3  Develop  Objectives  for  New  Tasks. 

i.  Task  Vs.  Learning  Analyses.  You  must  develop  training  objec¬ 
tives  for  the  new  tasks  introduced  to  the  task  list  and  for  the  tasks 
that  were  substantially  modified.  According  to  SAT,  training  objectives 
are  developed  from  a  learning  analysis  of  the  tasks.  The  learning  anal¬ 
ysis,  in  turn,  is  derived  from  a  task  analysis.  Both  the  task  and 
learning  analyses  detail  the  conditions,  actions,  and  standards  of  task 
performance  along  with  the  important  skills  and  knowledges  one  must 
possess  to  perform  the  task.  The  critical  difference  between  the  two 
analyses  is  that  the  task  analysis  is  stated  from  the  point  of  view  of 
the  job,  whereas  the  learning  analysis  takes  the  constraints  of  the 
training  environment  into  account.  The  purpose  of  the  learning  analysis 
is  to  act  as  a  "bridge"  between  actually  performing  the  task  and  learn¬ 
ing  how  to  perform  the  task  (Vineberg  &  Joyner,  1980).  In  practice, 
however,  the  two  types  of  analyses  are  very  similar.  The  detailed 
description  of  conditions,  actions,  standards,  skills,  and  knowledges 
represents  a  good  deal  of  effort  and  should  not  be  duplicated  over 
different  forms.  In  addition,  differences  between  performance  on  the 
job  and  performance  during  training  should  be  highlighted  instead  of 
obscured  In  the  details  of  the  two  analyses.  For  these  reasons,  the 
task  and  learning  analyses  should  be  combined  into  one  analysis  and  any 
crucial  differences  between  performing  and  learning  a  task  should  be 
noted  in  the  analysis  itself. 

b.  Combined  Analysis.  In  addition  to  detailing  the  task  data 
discussed  above,  task  and  learning  analyses  also  provide  other  types  of 
information.  Both  task  and  learning  analyses  provide  basic  task 
documentation  information  such  as  task  title,  task  number,  and  the 
references  used  for  the  analysis.  In  addition,  the  task  analysis 
provides  important  information  on  the  job  context  of  task  performance 
such  as  equipment  required,  personnel  required,  and  unique  working 
conditions  that  affects  task  performance.  On  the  other  hand,  the 
learning  analysis  also  provides  information  which  training  materials 
developers  will  need  such  as  recommended  training  activities,  recom¬ 
mended  media,  and  estimates  of  the  time  required  to  train  the  task.  The 
combined  task/learning  analysis  should  include  all  of  this  additional 
information. 

c.  Example .  To  illustrate  how  a  combined  task/learning  analysis 
may  be  performed,  a  sample  analysis  worksheet  is  shown  in  Figure  6-1  at 
the  end  of  the  chapter.  Many  of  the  items  were  modified  from  the  task 
analysis  worksheets  and  the  learning  analysis  worksheets  presented  In 
the  Training  Development  Handbook  (Draft  document,  USAARMC,  1980).  The 
example  task  entitled  "Prepare  and  Issue  Oral  Operation  Orders"  Is  a 
revised  task  which  was  suggested  for  inclusion  in  19K  BNCOC  (Morrison, 
Drucker,  &  O'Brien,  1985).  Currently  regarded  as  a  platoon  sergeant 
task,  the  course  revisers  proposed  that  the  task  be  modified  to  be 
relevant  to  tank  commander  orders  to  his  crew.  Each  of  the  18  items  of 
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the  analysis  are  explained  below  in  the  order  in  which  they  appear  in 
the  example  analysis  at  the  end  of  the  chapter. 

(1)  Task  Documentation  Data.  The  first  group  of  items  are 
essential  for  documenting  the  analysis. 

(a)  Date  of  Analysis.  Record  the  date  that  the  analysis 

is  finished. 

(b)  MOS  with  Skill  Level.  This  item  should  be  self- 

explanatory. 

(c)  Title.  Record  the  title  that  you  propose  to  give 
the  task.  If  the  task  represents  a  modification  of  an  existing  task, 
list  the  previous  title  in  parentheses. 

(d)  Task  Number.  Record  the  task  number  (or  numbers  if 
a  modified  task  is  a  combination  of  tasks)  of  the  task  being  modified. 

(e)  References .  State  every  reference  you  used  to 
perform  the  analysis.  The  reference  should  provide  enough  information 
for  others  to  identify  and  locate  the  source.  This  information  will  be 
particularly  useful  to  those  who  will  be  developing  new  training 
materials . 

(2)  Conditions.  The  second  group  of  items  detail  the 
conditions  under  which  the  task  is  performed. 

(a)  Task  Setting.  Describe  the  setting  and  general 
conditions  surrounding  task  performance. 

(b)  Supplies,  Tools,  and  Equipment.  List  all  pieces  of 
hardware  that  are  needed  to  perform  the  task. 

(c)  Job  Aids,  Manuals,  or  Data.  List  all  documents  that 
a  job  incumbent  has  available  to  him  and  to  which  he  can  refer  during 
task  performance.  Examples  include  technical  manuals,  checklists,  GTA 
cards,  etc. 

(d)  Personnel.  Identify  the  following  personnel  who  are 
involved  in  task  performance: 

1_  the  job  title  of  the  person  who  actually  performs 
the  task  being  analyzed, 

2_  the  job  title(s)  of  anyone  who  assists  in 
performance  of  the  task  in  question,  and 

_3  anyone  who  assigns  the  task  to  the  jobholder,  who 
supervises  the  process,  or  who  approves  the  final  product  of  the  task. 
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(e)  Unique  Working  Conditions.  List  any  unique  working 
conditions  that  might  affect  task  performance.  The  key  word  to  this 
element  is  "unique."  Unique  conditions  include  unusual  physical 
requirements  (e.g.,  20-20  vision)  or  restrictive  working  conditions 
(e.g.,  cramped  position). 

(f )  Job  Conditions  Not  Replicable  for  Training. 

Describe  any  condition  that  cannot  be  created  for  training  because  of 
safety  reasons,  cost  considerations,  or  any  other  reason. 

(3)  Initiating  Cues.  Identify  the  cues  that  signal  the  job 
incumbent  to  execute  the  task.  Every  task  has  one  or  more  initiating 
cues  that  may  come  from  a  variety  of  internal  or  external  sources. 

(4)  Standards.  Specify  how  well  the  task  must  be  performed. 
Standards  of  performance  can  describe  the  adequacy  of  the  task  product, 
the  speed  and  accuracy  of  task  performance,  or  both  whenever  appropri¬ 
ate.  If  the  training  objective  is  broken  down  into  multiple  enabling 
objectives,  list  the  performance  standards  for  each  separately. 

(a)  On  the  Job.  Determine  the  standards  for  performance 
on  the  job.  You  can  sometimes  derive  performance  standards  from 
doctrine  or  from  technical  manuals.  If  such  information  cannot  be 
found,  consult  an  SME  in  order  to  determine  an  appropriate  standard. 

(b)  During  Training.  Determine  whether  or  not  training 
standards  should  differ  from  job  standards.  Training  standards  may 
differ  from  job  standards  for  a  variety  of  reasons.  For  instance, 
training  standards  may  be  less  than  job  standards  because  time,  cost,  or 
safety  considerations  do  not  allow  enough  practice  on  the  task  to  attain 
job  standards.  Presumably,  additional  on-the-job  training  would  be  pre¬ 
scribed  in  this  case  to  bring  the  job  incumbent  up  to  standards. 
Alternatively,  training  standards  may  be  greater  than  job  standards  if 
on-the-job  training  is  impractical  and  if  rapid  skill  degradation  is 
expected.  However,  in  most  cases,  it  is  desirable  for  training  stand¬ 
ards  to  match  job  standards. 

(5)  Actions .  To  describe  the  action  component  of  the 
training  objective,  the  behaviors  are  broken  down  Into  separate 
elements.  Skills/knowledges  and  notes  are  listed  alongside  the  appro¬ 
priate  task  element. 

(a)  Elements.  List  all  the  steps  the  job  incumbent  has 
to  perform  in  executing  the  task.  If  the  objective  can  be  broken  down 
into  enabling  objectives,  use  an  outline  form  to  show  the  hierarchical 
structure.  That  is,  use  arabic  numerals  (1,  2,  3  .  .  .)  to  indicate 
enabling  objectives  and  letters  (a,  b,  c  .  .  .)  for  the  performance 
elements. 


(b)  Skills  and  Knowledges.  Skills  and  knowledges  refer 
to  the  so-called  "mental"  aspects  of  task  performance.  Identify  these 
components  and  locate  them  beside  the  appropriate  task  elements. 
Instruction  may  or  may  not  be  developed  for  each  skill/knowledge 
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component  depending  on  your  best  estimate  of  whether  or  not  students 
possess  those  components  upon  entering  the  course  (see  Chapter  3). 

(c)  Notes.  In  this  column,  provide  any  important 
information  that  does  not  fit  into  any  other  category.  This  sort  of 
information  might  include:  points  that  need  further  clarification,  how 
the  corresponding  training  element  may  differ  from  the  job  element;  etc. 

(6)  Training  Requirements.  In  this  section,  the  analyst 
provides  information  that  can  be  used  to  estimate  the  training  require¬ 
ments  of  the  task.  It  should  be  emphasized  the  following  information  is 
really  nothing  more  than  an  estimate  of  the  training  requirements 
proposed  in  a  relatively  early  stage  of  course  revision. 

(a)  Learning  Activities.  Briefly  outline  suggested 
learning  activities  for  attaining  the  training  objective.  The  ISD 
procedures  provide  guidelines  for  choosing  appropriate  learning 
activities.  However,  the  guidelines  are  based  on  a  taxonomy  of  learning 
objectives  that  is  not  universally  accepted  among  instructional 
developers.  The  specif ication  of  learning  activities  is  usually  left  to 
those  who  write  lesson  plans.  Thus,  this  item  should  be  the  analyst's 
best  guess  which  is  regarded  as  only  a  suggestion  to  those  developing 
the  training  materials. 

(b)  Media/Equipment .  Estimate  the  types  of  equipment 
needed  to  support  the  learning  activities.  Be  sure  to  Include  special 
training  devices  or  media  that  may  be  applicable  to  training  the  task. 
Like  the  previous  entry  (learning  activities),  this  entry  should  only  be 
regarded  as  a  suggestion  to  the  training  developer.  Nevertheless,  two 
publications  that  relate  to  this  issue  may  help  make  your  suggestions 
more  informed: 


1_  If  one  of  the  primary  reasons  for  revising  the 
course  is  to  incorporate  new  training  technologies,  you  should  construct 
an  inventory  of  applicable  technologies  and  identify  the  tasks  to  which 
they  apply.  Such  an  analysis  was  conducted  for  19K  BNCOC  as  described 
in  Part  II  of  Drucker,  Hannaman,  Melching,  and  O'Brien  (1984).  By 
example,  this  report  provides  possible  information  sources  and  methods 
for  this  sort  of  analysis. 

_2  The  Army  Research  Institute  has  published  a 
guidebook  for  selecting  appropriate  training  media  (Reiser,  Gage,  Wager, 
Larsen,  Hewlett,  Noel,  Winner,  &  Fagan,  1981).  The  guide  provides  some 
useful  insights  in  the  process  of  media  selection.  However,  like  the 
procedures  for  prescribing  learning  activities,  the  media  selection 
process  is  based  on  the  same  sort  of  learning  taxonomy  that  is  not  uni¬ 
versally  accepted. 

(c)  Personnel .  List  the  number  of  instructors, 
assistant  instructors,  controllers,  and  any  other  personnel  that  you 
i  think  will  be  needed  to  carry  out  the  instruction. 
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(d)  Time.  Estimate  the  time  required  to  train  the  task. 
If  possible,  break  the  time  down  into  that  required  for  initial 
classroom  instruction  and  that  required  for  any  practical  exercises. 

(7)  Statement  of  the  Training  Objective.  Using  the  criteria 
for  evaluating  objectives  (para  6-2d  above)  and  the  information  from 
this  analysis,  state  the  training  objective  in  paragraph  form.  Be  sure 
to  include  clear  statements  of  task  actions,  conditions,  and  standards. 
Use  the  acceptable  training  objectives  for  old  tasks  as  models. 

(8)  Evaluation  Criteria.  Determine  exactly  what  performance 
criteria  will  be  used  as  evidence  that  the  training  objective  has  been 
attained.  As  above,  be  sure  to  state  the  criteria  in  terms  of  task 
actions,  conditions,  and  standards.  The  difference  between  the  last  two 
elements  is  that  the  former  constitutes  guidance  for  training  develop¬ 
ment,  whereas  the  latter  relates  to  test  development.  Ideally  the 
training  objective  and  the  evaluation  criteria  will  be  identical. 
However,  because  of  time  and  resource  constraints  in  the  testing 
situation,  they  may  differ.1 


1The  apparent  differences  between  the  training  objective  and  the  evalu¬ 
ation  criteria  for  the  present  example  task  are  discussed  in  Chapter  8. 
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ACTIONS 


Example  Task/Learning  Analysis  (Continued) 


Example  Task/Learning  Analysis  (Continued) 


SKILLS  AND  KN 


Figure  6-1.  Example  Task/Learning  Analysis  (Continued) 


Example  Task/Learning  Analysis  (Continued) 


CHAPTER  7 


ORGANIZE  TRAINING  OBJECTIVES 


7-1.  Introduction. 


a.  Now  that  the  training  objectives  have  been  developed,  you  must 
structure  and  sequence  them  in  some  meaningful  way.  The  purpose  of  this 
phase  of  course  revision  is  to  do  just  that,  i.e.,  to  organize  the 
training  objectives  into  manageable  blocks  of  instruction.  This  organ¬ 
ization  has  at  least  three  positive  effects  on  training  effectiveness: 

(1)  First,  the  proper  organization  of  course  objectives 
enhances  trainee  performance  by  maximizing  the  transfer  of  skills  and 
knowledges  from  one  subject  area  to  another  and  by  the  increased  mean¬ 
ingfulness  provided  by  course  structure. 

(2)  Second,  it  also  increases  the  efficiency  of  course  admin¬ 
istration  by  eliminating  duplication  in  instruction  and  by  making 
effective  use  of  training  facilities  and  equipment. 

(3)  Third,  the  proper  organization  of  training  objectives 
facilitates  course  development  by  dividing  the  course  into  meaningful 
units  that  can  be  developed  independently  of  other  units. 

b.  According  to  IPISD  procedures,  the  proper  sequencing  and  clus¬ 
tering  of  training  objectives  is  determined  by  an  analysis  of  the 
relationships  between  objectives.  For  instance,  objectives  might  be 
related  by  virtue  of  the  fact  that  one  is  a  prerequisite  to  the  other 
(called  a  dependent  relationship)  or  that  two  objectives  are  performed 
in  close  temporal  proximity  on  the  job  (a  so-called  supportive  relation¬ 
ship).  In  practice,  such  an  analysis  may  be  an  unrealistic  requirement. 
For  instance,  for  a  relatively  short  list  of  50  training  objectives, 
there  are  over  1000  possible  relationships  between  tasks.  Furthermore, 
Vineberg  and  Joyner  (1980)  criticized  the  decisions  rules  for  structur¬ 
ing  and  sequencing  on  the  basis  of  these  relationships.  They  argued 
that  the  decision  rules  are  often  incomplete  and.  In  some  cases,  con¬ 
flicting.  Results  from  their  field  study  of  training  projects  indicated 
that  training  developers  did  not  analyze  all  task  interrelationships  to 
actually  sequence  and  cluster  objectives.  Instead,  the  organization  of 
objectives  was  primarily  determined  by  the  availability  of  training 
facilities  and  equipment. 

c.  To  develop  a  more  practicable  procedure  for  organizing  train¬ 
ing  objectives,  it  should  first  be  recognized  that  there  is  considerable 
Inherent  structure  in  the  course,  especially  by  the  time  you  have  gotten 
this  far  in  the  process  of  course  revision.  At  the  lower  level  of 
organization,  the  previous  analysis  of  the  various  task  components 
(subtasks  and  individual  elements)  provides  structure  and  sequence  to 
instruction  within  an  individual  training  objective.  At  a  higher  struc¬ 
tural  level,  the  tasks  have  already  been  divided  into  functional  areas. 
The  proposed  procedure  is  designed  to  utilize  the  existing  structure  and 
create  a  level  of  organization  that  Is  midway  between  those  extremes. 
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That  is,  it  will  indicate  which  objectives  need  to  be  combined  to  create 
logical  blocks  of  instruction. 

7-2.  Determine  Method  for  Sorting  Objectives.  Because  there  are  no 
hard  and  fast  rules  for  clustering  and  sequencing  objectives,  there  are 
numerous  ways  that  objectives  can  be  reasonably  organized.  You  must  be 
able  to  try  out  several  alternative  organizations  and  then  judge  for 
yourself  which  is  the  best  organization.  Consequently,  you  will  need  an 
efficient  method  for  sorting  and  resorting  objectives.  The  suggested 
method  is  to  write  the  task  title  corresponding  to  each  training  objec¬ 
tives  on  separate  index  cards.  Cards  can  be  shuffled  and  reshuffled 
into  various  orders.  Clusters  of  objectives  can  be  indicated  by  paper 
clipping  two  or  more  cards  together. 

7-3.  Establish  Functional  Categories.  Part  of  the  problem  with  the 
IPISD  procedure  for  sequencing  and  clustering  objectives  is  that  train¬ 
ing  developers  must  consider  all  the  objectives  at  once.  If  done  in 
this  manner,  the  procedure  for  clustering  and  sequencing  can  exceed  the 
limits  of  human  information  processing.  It  is  far  easier  to  work  within 
smaller  categories  of  objectives.  This  strategy  lessens  the  cognitive 
workload  by  reducing  the  number  of  items  that  the  developer  needs  to 
consider  at  once.  The  homogeneity  of  the  groups  also  makes  it  easier  to 
spot  commonalities  across  objectives 

a.  Review  Existing  Categories.  Functional  categories  were  estab¬ 
lished  during  the  revision  of  the  task  list.  These  categories  must  be 
reviewed  to  see  if  they  make  sense  for  training.  Using  a  19K  BNCOC 
example,  "Prepare  a  Sketch  Range  Card"  was  initially  identified  as  a 
gunnery  task  since  it  related  to  indirect  firing  techniques.  However, 
training  the  task  would  not,  and  probably  should  not,  require  live 
firing  that  is  central  to  gunnery  training.  The  other  consideration  was 
that  this  task  is  also  related  to  preparing  defensive  positions,  which 
is  typically  regarded  as  a  tactical  topic.  Consequently,  the  task  was 
moved  to  the  tactics  category  for  training  purposes. 

b.  Sort  Objectives  into  Categories.  Using  your  method  for  sort¬ 
ing,  place  each  objective  into  one  and  only  one  category.  The  following 
example  from  19K  BNCOC  is  the  category  of  tactics.  The  12  objectives 
(tasks)  in  this  category  include: 

(1)  Call  For  and  Adjust  Indirect  Fire 

(2)  Direct  Evasion  of  an  Enemy  Anti-Tank  Guided  Missile 

(3)  Estimate  Range 

(4)  Conduct  Target  Acquisition 

(5)  Direct  Reorganization  on  the  Objective 

(6)  Select  a  Firing  Position 

(7)  Prepare  a  Sketch  Range  Card 


(8)  Prepare  and  Issued  an  Oral  Operation  Order 

(9)  Prepare  a  Situation  Report  (SITREP) 

(10)  Conduct  a  Tactical  Road  March 

(11)  Maintain  Position  in  Platoon  Formation 

(12)  Prepare  and  Submit  a  Standard  Shelling,  Mortaring  and 
Bombing  Report 

7-4.  Cluster /Sequence  Objectives  Within  Categories.  Now  that  the 
objectives  are  divided  into  manageable  groups,  organize  the  objectives 
within  each  of  the  functional  categories.  The  following  instructions 
imply  that  one  should  cluster  the  objectives  first  and  then  sequence. 

In  actuality,  the  process  is  much  more  interactive.  New  ways  to 
sequence  will  lead  to  different  clustering  and  vice  versa. 

a.  Cluster  Objectives.  There  are  at  least  two  degrees  of 
relationship  between  objectives  which  may  be  identified: 

(1)  First  Order  Relationships.  These  refer  to  training 
objectives  that  are  so  closely  related  that  they  should  be  trained  in 
the  same  block  of  instruction.  Possible  first  order  relationships 
include  those  where  one  objective  is  a  prerequisite  to  another;  one 
objective  is  a  part  (i.e.,  a  subobjective)  of  another;  or  the  objectives 
share  many  of  the  same  attributes,  e.g.,  the  conditions,  actions, 
standards,  etc.  Because  first  order  clusters  form  single  blocks  of 
instruction,  they  must  be  given  a  name  which  is  descriptive  of  the 
overall  objective.  Four  first  order  relationships  can  be  identified  in 
the  19K  BNCOC  example  of  tasks  in  the  functional  area  of  Tactics: 

(a)  Three  of  the  objectives  involve  the  preparation  of 
orders  and  reports:  "Prepare  and  Issue  an  Oral  Operation  Order"; 
"Prepare  a  Situation  Report  (SITREP)";  and  "Prepare  and  Submit  a 
Standard  Shelling,  Mortaring,  and  Bombing  Report."  Positive  transfer 
would  result  from  having  these  tasks  trained  in  close  proximity.  On  the 
other  hand,  negative  interference  between  similar  objectives  may  also 
occur.  Instruction  should  be  designed  to  minimize  the  detrimental 
effects  of  interference.  For  instance,  the  developer  should  allow  for 
practice  on  distinguishing  the  reports.  The  suggested  cluster  name  is 
"Prepare  Orders  and  Reports." 

(b)  Two  tasks  deal  with  tactical  movements  with  the 
tank:  "Conduct  a  Tactical  Road  March"  and  "Maintain  Position  in  Platoon 
Formation."  Though  not  a  formal  subtask,  "maintaining  position"  may  be 
considered  as  part  of  a  tactical  road  march.  These  objectives  should  be 
trained  together  in  a  cluster  named  "Participate  in  Coordinated  Platoon 
Movement . " 


(c)  Similar  to  the  previous  example,  "Estimate  Range"  is 
part  of  the  objective  entitled  "Conduct  Target  Acquisition."  Because 
the  first  is  subordinate  to  the  second  objective,  the  cluster  should 
also  be  named  "Conduct  Target  Acquisition." 
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(d)  "Select  a  Firing  Position"  and  "Prepare  a  Sketch 
Range  Card"  are  related  because  they  are  both  components  of  preparing  a 
defensive  position.  Hence,  the  corresponding  cluster  should  be  named 
"Prepare  Defensive  Positions." 

(2)  Second  Order  Relationships.  Second  order  relationships 
include  objectives  that  are  similar  at  least  in  superficial  aspects,  but 
not  similar  enough  to  be  clustered  within  the  same  block  of  instruction. 
Objectives  having  a  second  order  relationship  should  nevertheless  be 
placed  close  together  in  the  course  to  maximize  skill  transfer.  If  the 
objectives  share  similar  conditions,  they  will  likely  require  similar 
training  equipment  and  facilities.  Two  examples  of  these  relationships 
can  be  drawn  from  19K  BNCOC: 

(a)  "Direct  Evasion  of  an  Enemy  Anti-Tank  Guided 
Missile"  is  related  to  the  two  movement  tasks  in  the  cluster  entitled 
"Participate  in  Coordinated  Platoon  Movement."  Even  though  the  task 
involves  neither  tactical  nor  coordinated  movement  per  se,  it  should  be 
placed  close  to  the  other  two  tasks  because  training  on  both  tasks 
requires  either  a  moving  tank  or  training  equipment  that  simulates 
movement . 


(b)  "Call  For  and  Adjust  Indirect  Fire"  shares  task 
components  with  "Estimate  Range"  and  "Conduct  Target  Acquisition."  All 
three  should  be  in  close  proximity,  but  the  indirect  fire  task  deserves 
a  separate  block  of  instruction  because  of  its  complexity. 

b.  Sequence  Objectives.  After  potential  clusters  have  been 
identified,  objectives  are  then  sequenced  both  within  and  among  the 
clusters. 


(1)  Within  Clusters.  There  are  many  ways  that  objectives 
within  clusters  may  be  sequenced.  The  following  suggested  rules  for 
sequencing  were  taken  from  the  Training  Development  Handbook  (Draft 
document,  US  Army  Armor  School,  1980,  p.  2-22): 

(a)  Job  Performance  Order.  Objectives  should  be  trained 
in  the  same  order  that  they  are  performed  on  the  job.  Using  the 
"Prepare  Defensive  Position"  as  an  example,  the  objective  of  "Select  a 
Firing  Position"  should  be  trained  before  "Preparing  a  Sketch  Range 
Card"  because  a  tank  must  first  be  in  a  firing  position  before  a  crewman 
is  able  to  sketch  a  range  card. 

(b)  Chronological  Order.  Similar  to  but  more  general 
than  the  previous  guideline,  this  rule  states  that  objectives  corres¬ 
ponding  to  certain  events  should  be  trained  in  the  same  order  as  they 
occur.  In  the  "Prepare  Orders  and  Reports"  cluster,  the  suggested  order 
of  objectives  ("Prepare  and  Issue  an  Oral  Operation  Order";  "Prepare  a 
Situation  Report  (SITREP)";  and  "Prepare  and  Submit  a  Standard  Shelling, 
Mortaring,  and  Bombing  Report")  corresponds  to  events  that  generally 
occur  before,  during  and  after  battle.  Similarly,  within  the  cluster 
entitled  "Participate  in  Coordinated  Platoon  Formation,"  the  task 
entitled  "Maintain  Position  in  Platoon  Formation"  occurs  after  platoon 
movement  has  been  initiated.  Thus,  it  should  be  trained  after  "Conduct 
Tactical  Road  March,"  which  provides  for  initiating  the  movement. 
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(c)  Cause  and  Effect.  Instruction  on  the  causes  of  an 
event  should  precede  a  description  of  the  effects  of  an  event.  This 
rule  would  appear  to  apply  to  sequencing  elements  within  knowledge 
oriented  objectives.  The  example  taken  from  the  Armor  School's  handbook 
is  that  an  explanation  of  the  causes  of  unit  morale  problems  should  be 
presented  prior  to  a  description  of  the  effects  of  the  problem  on  unit 
performance . 


(d)  Critical  Sequence.  Any  critical  sequence  in  a 
procedure  must  be  trained  in  the  order  the  sequence  occurs.  As  in  the 
previous  guideline,  this  rule  appears  to  apply  to  sequencing  the 
elements  within  training  objectives.  For  instance,  in  order  to  charge  a 
machinegun,  the  safety  must  be  off  before  pulling  the  bolt  to  the  rear 
in  order  to  prevent  damage  to  the  gun.  Thus,  students  must  first 
receive  training  on  putting  the  safety  in  the  FIRE  position  prior  to 
being  trained  to  pull  the  bolt  to  the  rear. 

(e)  Simple  to  Complex.  According  to  this  guideline, 
simple  objectives  should  be  trained  prior  to  complex  ones.  Using  the 
19K  BNCOC  example,  the  relatively  simple  task  entitled  "Estimate  Range" 
should  be  trained  before  the  more  complex  task  called  "Conduct  Target 
Acquisition. " 


(f)  Known  to  Unknown.  Familiar  topics  should  be  handled 
before  unfamiliar  ones.  Again  using  the  19K  BNCOC  example,  the  tank 
commander  trainees  should  be  very  familiar  with  the  task  entitled 
"Conduct  Target  Acquisition,"  at  least  from  the  gunner's  point  of  view. 
However,  they  probably  have  had  little  or  no  experience  on  the  related 
task  termed  "Call  For  and  Adjust  Indirect  Fire."  Thus,  the  former 
should  precede  the  latter. 

(2)  Among  Clusters.  The  clusters  themselves  must  now  be 
sequenced.  The  result  of  this  step  is  that  the  order  of  objectives 
within  a  functional  category  should  be  completely  resolved.  The  rules 
for  sequencing  within  cluster  apply  also  to  sequencing  among  clusters. 

An  additional  guideline  that  can  be  used  for  sequencing  among  clusters 
is  to  follow  a  tactical  scenario.  This  rule  is  used  to  sequence  the 
tactical  objectives  in  the  19K  BNCOC  example.  The  scenario  starts  with 
preparation  to  move  out.  During  movement,  the  tank  encounters  enemy 
targets  in  the  process  of  obtaining  an  objective.  As  a  result  of 
battle,  the  tank  commander  has  to  reorganize  his  crew  and  equipment. 
Finally,  the  tank  commander  secures  the  objective  by  preparing  a  defen¬ 
sive  position.  Following  this  scenario,  the  clusters  and  objectives  in 
the  tactical  functional  area  are  sequenced  as  follows: 

(a)  Prepare  Orders  and  Reports: 

^  Prepare  and  Issue  an  Oral  Operation  Order 

Prepare  a  Situation  Report  (SITREP) 

3_  Prepare  and  Submit  a  Standard  Shelling  Mortaring, 

and  Bombing  Report 
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(b)  Participate  in  Coordinated  Platoon  Movement 


_1.  Conduct  a  Tactical  Road  March 
1_  Maintain  Position  in  Platoon  Formation 

(c)  Direct  Evasion  of  an  Enemy  Anti-Tank  Guided  Missile 

(d)  Conduct  Target  Acquisition 
1_  Estimate  Range 

2_  Conduct  Target  Acquisition 

(e)  Call  For  and  Adjust  Indirect  Fire 

(f)  Direct  Reorganization  on  the  Objective 

(g)  Prepare  Defensive  Position 

Select  a  Firing  Position 
2_  Prepare  a  Sketch  Range  Card 

c.  Verify  Organization  within  Categories.  As  Vineberg  and  Joyner 
(1980)  point  out,  the  various  rules  used  for  clustering  and  sequencing 
are  incomplete,  and  in  some  cases,  in  conflict  with  one  another.  Con¬ 
sequently,  there  is  no  one  single  best  organization  of  objectives  within 
categories.  The  problem  is  particularly  acute  if  you  know  very  little 
about  the  category  in  question.  If  you  are  unsure  about  your  organiza¬ 
tion  of  the  objectives  within  a  category,  have  a  few  SMEs  go  through  the 
same  exercise  as  you  did.  You  should  furnish  them  the  cards  you  used 
and  the  guidelines  for  organizing  objectives,  but  do  not  influence  them 
in  any  other  way.  If  you  have  more  than  one  SME  doing  this  procedure, 
be  sure  they  work  independently.  You  will  most  likely  find  that  SMEs 
will  agree  on  certain  segments  of  the  course  and  disagree  on  others. 

Your  final  arrangement  should  reflect  your  own  evaluation  of  SMEs' 
responses  as  well  as  majority  opinion  on  how  objectives  should  be  organ¬ 
ized.  Even  though  a  majority  of  SMEs  may  arrange  objectives  a  certain 
way,  you  may  wish  to  follow  the  arrangement  of  a  single  SME  who  provides 
a  convincing  rationale  for  his  particular  arrangement. 

7-5 .  Sequence  Functional  Areas. 

a.  After  the  objectives  have  been  sequenced  within  each  func¬ 
tional  area,  you  must  sequence  the  functional  areas  themselves.  Again, 
the  rules  and  guidelines  for  sequencing  may  apply  to  sequencing  func¬ 
tional  areas.  However,  at  this  level  of  organization,  an  additional 
guideline  for  sequencing,  called  the  "Crawl-Walk-  Run"  principle,  may 
come  into  play.  This  principle  asserts  that  in  order  to  learn  how  to 
run,  first  you  have  to  learn  how  to  crawl,  and  then  to  walk.  In  other 
words,  instruction  should  be  sequenced  so  that  the  student  starts  in 
simple  learning  situations  that  are  contrived  to  allow  him/her  to 
acquire  some  of  the  most  basic  skills.  Instruction  then  proceeds  to 
ever-increasing  levels  of  skill  integration  and  task  realism. 


b.  The  crawl-walk-run  principle  was  applied  to  19K  BNCOC.  The 
revised  course  can  be  conceived  as  consisting  of  three  parts. 

(1)  In  the  first  part,  the  most  abstract  functional  areas  are 
presented  primarily  in  a  classroom  environment.  The  functional  areas 
(in  sequence)  are: 

(a)  Leadership  and 

(b)  Training. 

(2)  The  second  part  presents  content  areas  that  are  related 
but  not  necessarily  unique  to  armor.  Instruction  is  a  combination  of 
classroom  and  field  work.  The  functional  areas  in  the  second  part  are: 

(a)  Nuclear,  Biological,  and  Chemical  Defense; 

(b)  Mine  Warfare; 

(c)  Communications;  and 

(d)  Land  Navigation. 

(3)  The  third  part  is  armor-specific  and  involves  a  lot  of 
on-tank  work:  Maintenance,  Gunnery,  and  Tactics.  It  is  significant  to 
note  that  the  subject  of  Gunnery  occurs  before  Tactics,  which  is  in 
contrast  to  the  sequence  in  the  existing  course.  The  rationale  for  this 
sequence  in  the  revised  course  was  that  the  subject  of  Tactics  inte¬ 
grates  all  of  the  functional  areas  and  should  therefore  be  placed  last 
in  the  sequence.  The  proposed  order  for  the  last  part  of  the  course  is: 

(a)  Maintenance, 

(b)  Tank  Gunnery,  and 

(c)  Tactical. 


CHAPTER  8 


DEVELOP  TESTS 


8-1.  Introduction. 

a.  There  are  two  reasons  for  developing  tests  at  this  point  in 
the  course  revision  process.  The  first  is  that  tests  must  be  developed 
prior  to  developing  instruction.  According  to  the  logic  of  ISD  models, 
tests  are  developed  from  training  objectives  (as  opposed  to  being 
developed  from  instructional  content)  because  they  are  intended  to 
measure  student  performance  on  those  objectives.  In  addition,  having 
completed  tests  prior  to  the  development  of  training  helps  to  ensure 
that  the  course  reviser  develops  instruction  that  pertains  to  the 
training  objectives  and  not  to  extraneous  instructional  material. 

b.  The  second  reason  for  placing  test  development  at  this  point 
is  that  tests  must  be  developed  after  training  objectives  have  been 
clustered  and  sequenced.  This  allows  tests  to  be  developed  on  the  basis 
of  meaningful  blocks  of  instruction  and  not  individual  objectives.  If 
clustering  and  sequencing  were  to  occur  after  test  development,  the 
tests  would  most  likely  have  to  be  reorganized  to  correspond  with  the 
organization  of  instruction. 

8-2.  Review  Tests  for  Old  Tasks. 


a.  Secure  tests  for  tasks  that  are  in  the  present  POI.  You  may 
have  to  make  some  minor  changes  in  organization  if  the  tasks  are  not 
clustered  the  same  as  in  the  proposed  course  revision. 

b.  Tests  should  be  reviewed  and  revised  In  response  to  two 
general  questions: 

(1)  What  is  the  correct  mode  for  testing  the  objective?  This 
issue  is  discussed  in  detail  below  (para  8~3a) . 

(2)  Does  the  test  provide  an  accurate  measure  of  whether  or 
not  a  student  has  attained  the  objective?  Another  way  to  phrase  this 
question  is  can  you  predict,  from  test  performance,  whether  or  not  the 
student  can  actually  perform  the  task  in  question?  Although  this 
question  can  be  answered  by  actually  testing  soldiers,  you  will  probably 
not  have  the  time  and  resources  to  accomplish  an  empirical  determina¬ 
tion.  Therefore,  use  your  own  judgment  as  to  the  appropriateness  of  the 
test. 


8-3.  Develop  Tests  for  New  Tasks.  Much  of  the  following  material  is 
adapted  from  Guidelines  for  Development  of  Skill  Qualification  Tests 
(US  Army  Training  Support  Center,  December,  1977).  Although  these 
guidelines  concern  the  development  of  SQTs,  much  of  the  manual  is  appli¬ 
cable  to  development  of  tests  for  training.  You  are  encouraged  to  refer 
to  this  document  for  more  details  about  test  development  as  needed. 
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a.  Choose  Appropriate  Test  Mode. 


(1)  Types  of  Tasks.  In  order  to  choose  the  appropriate  mode 
of  testing  for  a  particular  objective,  you  must  characterize  the  corres¬ 
ponding  task  on  two  dimensions: 

(a)  Physical  Vs.  Mental.  Ask  yoursel  ices  the  task 
performance  primarily  require  physical  activity  (movement  of  the  body 
and  limbs)  or  does  it  mainly  require  thought?  Because  the  focus  is  on 
testing  at  this  point,  the  distinction  between  physical  and  mental  tasks 
depends  on  what  you  would  expect  to  observe  if  you  were  a  tester.  If 
you  would  expect  to  see  mostly  activity,  then  the  task  is  a  physical 
one.  If,  on  the  other  hand,  most  of  the  action  goes  on  in  the  student's 
head,  then  the  task  should  be  classified  as  mental. 

(b)  High  Vs.  Low  Skill.  Regardless  of  whether  or  not  a 
task  is  physical  or  mental,  ask  yourself  how  much  practice  a  soldier 
needs  to  execute  the  task.  If  he  can  be  expected  to  perform  the  task 
after  reading  about  or  being  told  about  the  task  with  no  more  than  a 
single  practice  trial,  you  may  characterize  the  task  as  requiring  low 
skill.  If,  in  contrast,  the  task  requires  repeated  practice  in  order 
for  the  soldier  to  execute  it  efficiently,  the  task  requires  a  high 
level  of  skill. 

(2)  Types  of  Tests.  There  are  essentially  two  types  of  tests 
which  you  can  develop  for  an  objective: 

(a)  Hands-on  Tests:  Soldiers  are  tested  by  performing 
the  task  on  actual  equipment. 

(b)  Written  Tests:  Soldiers  must  read  questions  and 
respond  in  writing. 

(3)  Choose  Appropriate  Test/Task  Combination.  Hands-on  tests 
can  be  developed  for  all  types  of  tasks.  Written  tests  are  probably 
appropriate  for  all  tasks  except  high-skill,  physical  ones.  When 
choosing  the  test  mode,  you  have  to  make  the  following  tradeoffs: 

(a)  Hands-on  tests  are  relatively  easy  to  develop  in 
that  they  involve  fairly  straightforward  translations  of  task/learning 
elements  into  performance  measures.  On  the  other  hand,  they  may  be 
expensive  to  administer  in  terms  of  accurately  reproducing  job  condi¬ 
tions.  And,  in  most  situa;ions,  hands-on  testing  can  only  be 
administered  individually. 

(b)  Written  tests  can  be  difficult  to  develop  in  that 
you  must  conceive  of  written  test  items  that  predict  how  well  soldiers 
will  perform  the  task.  Therefore,  development  of  appropriate  test  items 
may  take  time  and  require  a  good  deal  of  creativity.  On  the  other  hand, 
written  tests  are  relatively  cheap  to  administer  in  that  they  do  not 
normally  require  special  equipment  and  can  sometimes  be  administered  on 
a  group  basis. 
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8-4.  Develop  Hands-on  Tests.  The  central  issue  in  developing  hands-on 
tests  is  how  performance  is  to  be  measured.  There  are  basically  two 
methods  for  measuring  performance:  One  is  to  measure  the  product 
(outcome)  of  task  execution;  the  other  is  to  measure  the  process  of  task 
performance.  To  distinguish  the  two  concepts,  again  imagine  a  soldier 
actually  performing  the  task.  If  you  would  have  to  observe  the  entire 
performance  in  order  to  fairly  evaluate  it,  you  would  be  concerned  with 
process  measures.  If,  on  the  other  hand,  you  need  only  to  observe  the 
result  of  task  performance,  you  would  need  what  are  called  product 
measures  of  performance. 

a.  Product  Measurement.  Of  the  two  types  of  hands-on  measures, 
product  measures  are  more  desired  in  that  the  outcome  is  easier  to  score 
and  in  that  the  tester  does  not  need  to  vigilantly  attend  to  task  per¬ 
formance.  Consequently  scoring  procedures  are  more  easily  standardized 
between  testers.  Note  that  "products"  may  not  be  tangible;  they  may  be 
an  action  or  results  from  an  action.  For  instance,  for  a  radio  check 
task,  a  correctly  operating  radio  is  a  valid  product  measure.  To 
develop  appropriate  product  measures,  proceed  as  follows: 

(1)  Define  Acceptable  Product.  The  definition  of  an 
acceptable  product  ought  to  be  implicit  in  the  training  objective. 
Elaborate  on  the  standards  if  more  detail  is  needed  for  scoring  pur¬ 
poses.  Sometimes  the  standard  is  stated  in  terms  of  a  tolerated  range 
of  performance.  This  range  should  be  stated  in  the  definition  of  the 
product.  Examples  include  "locate  position  within  50  meters"  and 
"determine  direction  to  plus  or  minus  2  mils." 

(2)  Specify  Time  Limits. 

(a)  Some  tasks  have  inherent  time  constraints  that  are 
implicit  in  the  performance  standards.  Examples  include  "acquire  and 
engage  target  within  10  seconds."  Be  sure  such  constraints  are  speci¬ 
fied.  Any  time  limits  that  are  less  than  one  minute  should  be  timed  by 
a  reliable  stopwatch. 

(b)  Even  if  tasks  do  not  have  inherent  time  constraints, 
some  limitation  must  be  put  on  the  test  for  purposes  of  test  adminis¬ 
tration.  This  second  sort  of  time  limit  should  be  sufficiently  long  to 
allow  students  to  complete  the  test,  and  only  cut  off  those  who  would 
likely  have  failed  the  test  for  other  reasons.  As  a  general  guideline, 
the  time  limit  for  a  single  testing  station  should  be  no  longer  than  20 
minutes  at  the  maximum. 

(3)  Preserve  Product.  Some  products  are  not  long  lasting, 
and  must  be  preserved  in  order  for  the  tester  to  measure  it.  An  example 
product  which  may  not  last  is  a  sight  picture.  Appropriate  Instructions 
to  the  tester  should  tell  him  how  to  preserve  a  product  if  need  be. 

b.  Process  Measurement.  Process  measures  are  relatively  easy  to 
obtain  because  they  can  be  derived  directly  from  a  good  analysis  of  the 
task.  To  develop  appropriate  process  measures,  proceed  as  follows: 
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(1)  Ensure  that  Each  Element  Is  Obse*~vable.  Every  element  in 
the  test  must  be  observable.  A  particular  problem  in  that  regard  is  a 
checking  behavior.  A  behavior  such  as  "checking  to  see  if  the  safety  is 
in  the  SAFE  position"  may  not  be  an  observable  element.  You  could  drop 
the  element  from  the  test  if  it  is  nonessential  to  the  behavioral 
process.  However,  if  the  check  is  essential  or  a  required  safety 
measure,  the  testing  situation  must  be  somehow  contrived  so  that  the 
behavior  or  the  result  of  the  behavior  is  observable.  For  instance,  the 
equipment  shoulc  be  placed  such  that  the  check  requires  a  turn  of  the 
head  or  movement  of  the  limbs.  In  the  example  presented  above,  you 
could  instruct  the  tester  to  place  the  weapon  in  FIRE  before  the  test  in 
order  to  assure  that  the  student  must  make  an  overt  movement  as  a  result 
of  checking  the  weapon. 

(2)  Examine  the  Number  of  Elements.  The  task  steps  ought  to 
be  detailed  enough  so  that  testers  observe  behavioral  process  in  a 
standardized  manner.  On  the  other  hand,  too  many  elements  (more  than 
20)  on  a  scoresheet  can  overwhelm  the  tester.  If  the  analysis  lists 
many  elements,  some  may  be  eliminated  as  unimportant;  other  elements  may 
be  observed  and  not  scored. 

(3)  Determine  Whether  or  Not  Sequence  Is  Critical.  In  some 
tasks,  behavioral  elements  must  be  performed  in  a  set  sequence.  In 
others,  only  partial  sequence  or  no  sequence  is  required.  Identify 
which,  if  any,  part  of  the  task  must  be  performed  in  a  particular  order. 

(4)  Specify  Time  Limits  and  Tolerances.  The  considerations 
discussed  above  in  para  8-4a  apply  to  the  development  of  process 
measures. 


c.  Combination  Measures.  Many  tasks  do  not  fall  neatly  into 
either  product  or  process  measurement  category.  Some  aspects  of  the 
task  are  best  scored  by  product  measures;  others,  by  process  measures. 
The  best  strategy  is  to  use  both.  This  strategy  is  particularly  appro¬ 
priate  where  there  are  process  elements  of  the  task  that  are  not  evident 
in  the  final  product.  The  following  questions  ought  to  be  considered 
when  deciding  when  to  include  process  measures: 

(1)  Does  the  final  product  somehow  obscure  the  process?  In 
an  assembly  task,  for  instance,  the  exterior  of  an  assembled  weapon  may 
obscure  whether  or  not  the  components  were  correctly  assembled  in  the 
interior.  In  that  case,  some  process  measures  must  be  included. 

(2)  Are  there  safety  measures  to  consider?  An  extremely 
important  factor  in  test  development  is  whether  or  not  student  errors 
may  lead  to  injury  or  to  equipment  damage.  Often,  the  only  way  to 
monitor  appropriate  safety  related  behavior  is  through  process  measures. 
Safety-related  process  measures  must  be  included  in  any  test.  A  related 
issue  that  must  be  resolved  is  whether  to  stop  a  student  or  to  let  him 
continue  task  execution  once  he  has  committed  a  safety  violation.  This 
surely  depends  on  the  nature  of  the  violation;  hence  no  hard-and-fast 
rules  can  be  given  for  this  determination. 


(3)  Does  the  student  require  feedback  on  the  process  aspects 
of  the  task?  You  must  determine  whether  or  not  the  student  needs 
detailed  feedback  on  his  performance  in  order  to  determine  the  cause  of 
his  problem  after  failing  a  station.  Detailed  feedback  usually  requires 
attention  to  the  process  aspects  of  the  task. 

d.  Test  Conditions.  Once  the  performance  measures  are 
determined,  you  need  to  determine  the  conditions  under  which  the  student 
will  take  the  test.  The  rationale  for  this  procedure  is  to  specify 
important  conditions  that  may  affect  performance  and  to  control  for 
their  effect  by  standardizing  conditions  across  students.  The  testing 
conditions  ought  to  be  derived  from  the  statement  of  training  condi¬ 
tions.  However,  testing  conditions  are,  of  course,  more  specifically 
concerned  with  the  conditions  during  the  test  and  are  subject  to  the 
constraints  of  the  testing  situation.  You  need  to  examine  three  aspects 
of  test  conditions: 

(1)  Environment .  Specify  any  important  environmental  condi¬ 
tion  which  may  affect  behavior.  "Important"  is  underscored  to  emphasize 
that  not  every  condition  (e.g.,  exact  temperature,  ambient  light  levels, 
etc.)  needs  to  be  specified.  Example  important  environmental  conditions 
are  whether  testing  is  to  be  held  indoors  or  outdoors,  and  whether 
testing  will  be  conducted  on  a  group  or  individual  basis. 

(2)  Equipment.  Any  equipment  needed  in  the  test,  including 
manuals,  should  be  stated  as  part  of  the  testing  conditions.  Again,  you 
should  start  by  examining  the  equipment  requirements  as  stated  in  the 
training  objective. 

(3)  Station.  Determine  any  special  station  set-up  conditions 
that  need  to  be  specified.  For  instance,  equipment  controls  may  have  to 
be  placed  on  certain  settings  prior  to  testing  students. 

e.  Instructions  to  the  Tester.  One  of  the  important  differences 
between  hands-on  and  written  tests  is  the  relative  importance  of  the 
tester.  For  written  tests,  the  tester  acts  basically  as  proctor.  He 
hands  out  the  test,  and  perhaps  reads  some  instruction.  He  has  very 
little  else  to  do  while  the  test  is  being  taken.  For  a  hands-on  test, 
in  contrast,  the  tester  has  much  to  do  before  and  during  the  test.  In 
order  to  standardize  hands-on  tests,  the  testers  must  thoroughly  under¬ 
stand  their  role.  Consequently,  the  instructions  to  the  tester  are  very 
important.  At  a  minimum,  the  instructions  to  the  tester  should  cover 
the  following  points: 

(1)  Describe  how  the  station  should  be  set  up  for  the  first 
student  who  goes  through  testing  and  any  changes  that  must  be  made  for 
the  subsequent  students. 

(2)  State  in  step-by-step  fashion  how  the  equipment  should  be 
set  up  before  testing  any  students.  Also,  describe  any  maintenance  or 
readjustments  that  must  occur  between  students. 
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(3)  State  the  maximum  number  of  students  that  can  feasibly  be 
tested  at  one  point  in  time.  If  product  measures  are  being  used,  more 
than  one  student  may  be  tested  at  a  time.  An  important  consideration  in 
group  testing  is  the  extent  to  which  students  receive  cues  by  observing 
the  performance  of  other  students.  Your  instructions  should  include 
provisions  to  prevent  such  cueing,  e.g.,  minimum  distances  between 
students,  methods  for  blocking  visual  cues,  etc. 

(4)  Specify  any  time  limits  that  apply.  Include  a  descrip¬ 
tion  of  when  timing  should  start  and  when  it  should  end. 

(5)  Include  any  special  instructions  such  as  the  following: 

(a)  the  extent  of  assistance  allowed  during  task  per¬ 
formance; 

(b)  any  safety  considerations  that  require  the  tester  to 
interfere  with  test  performance;  and 

(c)  any  special  scoring  instructions  including  instruc¬ 
tions  on  how  to  use  aids  to  scoring. 

f.  Instructions  to  the  Student.  Another  difference  between 
hands-on  and  written  tests  is  that  the  instructions  for  hands-on  tests 
must  be  read  aloud.  Consequently,  you  should  write  these  Instructions 
with  the  understanding  that  they  are  to  be  read.  The  emphasis  should  be 
on  clarity  in  communication.  Do  not  use  acronyms  or  technical  terms 
unless  the  student  is  very  familiar  with  them.  Remember  that,  in  con¬ 
trast  to  written  instructions,  the  student  will  not  have  the  benefit  of 
rereading  the  instructions  more  closely  if  he  does  not  understand.  Test 
the  comprehensibility  of  the  Instructions  by  reading  them  aloud  to  a 
colleague.  Be  sure  the  instructions  cover  the  following  points: 

(1)  The  instructions  may  begin  with  a  brief  description  of 
the  job  situation.  The  description  may  be  a  translation  of  the  task 
conditions.  For  instance,  if  the  task  is  normally  performed  under 
combat  conditions,  the  instructions  may  tell  the  student  to  assume  that 
he  is  under  enemy  fire.  However,  it  should  also  be  noted  that  job 
situations  may  not  apply  or  may  not  be  needed  for  some  tasks. 

(2)  The  instructions  should  describe  to  the  student  what  is 
required  of  him  so  that  he  may  begin  the  task.  Be  careful  to  avoid 
either  overcueing  or  undercueing  here.  Overcueing  is  telling  the 
student  too  much,  i.e.,  telling  him  exactly  what  he  is  supposed  to  do. 
Undercueing  is  the  opposite  extreme:  The  student  has  too  little  infor¬ 
mation  to  perform  the  first  step  in  the  task  and  instructions  provide  no 
cues  as  to  subsequent  steps. 

(3)  State  any  time  or  accuracy  standards  that  apply  to  task 
performance. 

(4)  State  whether  or  not  assistance  will  be  allowed  and  the 
extent  of  the  assistance. 


51 


8-5.  Develop  Written  Tests.  In  order  to  develop  an  appropriate  written 
test,  you  need  to  consider  two  distinctions  used  in  test  development: 

a.  Test  Format.  The  first  distinction  relates  to  the  format  of 
the  written  test.  The  format  can  be  described  as  either  written  per¬ 
formance  or  performance-based.  It  is  likely  that,  in  actual  development 
of  a  written  test,  you  may  use  a  combination  of  written  performance  and 
performance-based  items.  Both  formats  are  described  below: 

(1)  Written  Performance.  This  sort  of  written  test  format 
requires  the  student  to  perform  the  task  as  he  would  on  the  job.  This 
format  should  be  used  whenever  task  performance  can  be  measured  as  a 
written  product.  The  format  for  a  written  performance  test  is  similar 
to  that  of  a  hands-on  test.  Hence,  many  of  the  same  guidelines  apply. 
Examples  of  tasks  for  which  written  performance  tests  can  be  developed 
include  identifying  coordinates  on  a  map,  filling  out  standard  forms, 
and  determining  range  to  a  target  using  the  mil  formula. 

(2)  Performance-Based. 

(a)  A  written  performance  test  should  be  developed  when¬ 
ever  product  measures  can  be  derived  from  written  behavior.  If  not,  you 
will  have  to  develop  a  performance-based  test,  A  performance-based  test 
evaluates  the  knowledge  that  a  student  must  possess  in  order  to  perform 
the  task.  In  format,  this  type  of  test  resembles  a  standard  academic 
exam. 

(b)  In  order  to  identify  those  knowledges  and  to  develop 
appropriate  test  items,  you  need  to  identify  reasons  why  a  soldier 
typically  fails  to  perform  an  element  in  a  task.  The  following  are  some 
"generic"  errors.  Review  each  task  element  with  respect  to  these  errors 
so  that  you  can  develop  appropriate  test  items: 

1^  Soldiers  do  not  know  where  to  locate  certain 
objects  or  positions  related  to  task  performance.  If  this  error  is 
applicable,  devise  a  test  question  wherein  the  student  must  identify  the 
correct  location. 

2^  Soldiers  do  not  know  when  in  a  sequence  of  task 
elements  to  perform  a  particular  step.  A  number  of  different  questions 
should  be  developed  to  test  the  student's  knowledge  of  sequence  of  the 
whole  task  or  a  particularly  troublesome  part  of  the  task. 

Soldiers  do  not  know  what  the  product  or  the  end 
result  of  an  element  should  look  like.  For  instance,  soldiers  cannot 
engage  targets  successfully  if  they  do  not  know  what  an  appropriate 
sight  picture  is.  Appropriate  test  items  for  this  type  of  error  may 
incorporate  detailed  descriptions  or  illustrations  of  the  product. 

4^  Soldiers  do  not  know  how  to  perform  a  procedure. 
Test  items  should  be  developed  depending  upon  the  nature  of  the  proce¬ 
dural  error. 


b.  Response  Format.  The  second  distinction  used  in  test  develop¬ 
ment  concerns  the  format  of  the  responses  that  the  student  makes  to 
answer  the  test  items.  There  are  two  types:  recognition  and  free 
response.  Although  both  types  of  questions  may  be  developed  for  either 
written  performance  or  performance-based  test  formats,  the  recognition 
response  is  probably  better  suited  for  performance-based,  whereas  the 
free  response  technique  is  better  for  written  performance  tests. 

(1)  Recognition  Responses. 

(a)  In  the  first  response  format,  the  student  is 
required  to  recognize  the  correct  response  from  a  list  of  alternative 
responses.  Examples  of  recognition  response  formats  include  multiple 
choice  and  matching.  Frequently  referred  to  as  an  "objective"  tech¬ 
nique,  this  response  format  has  the  advantage  of  being  easy  to  score 
with  essentially  no  problems  in  standardization.  If  appropriate  recog¬ 
nition  test  items  can  be  derived  for  a  task,  this  is  probably  the 
response  format  of  choice. 

(b)  By  far,  the  most  frequently  used  type  of  recognition 
test  is  the  multiple  choice  format.  The  validity  of  a  multiple  choice 
test  item  depends  largely  on  the  selection  of  appropriate  response 
alternatives.  The  alternatives  must  be  plausible.  That  is,  they  ought 
to  represent  incorrect  choices  that  soldiers  make  on  the  job.  The 
number  of  alternatives  ought  to  be  dictated  by  the  number  of  plausible 
responses.  Obviously,  there  are  some  limits  on  the  number  of  alterna¬ 
tives,  however.  There  must  be  at  least  one  alternative  to  the  correct 
answer,  and  ten  or  more  alternatives  may  be  unwieldy  to  answer  and  to 
score. 

(c)  For  a  written  performance  test,  the  test  standards 
should  be  derived  from  the  job/training  standards.  However,  the 
standard  is  less  clear  for  performance-based  test  items  because  they  are 
not  as  closely  related  to  job  performance.  If  the  standards  are  not 
clear,  use  the  following  two  standard  rules  that  are  used  in  SQT  devel¬ 
opment  : 

1^  Adopt  an  80%  criterion  for  passing  a  test.  For 
instance,  the  criterion  for  passing  a  4-item  test  is  .8X4“  3.2  or  3 
test  items,  rounded  to  the  nearest  whole  number. 

1_  Adopt  a  100%  criterion  for  passing  a  standard 
multiple  choice  item  where  only  one  alternative  is  correct.  That  is, 
the  student  must  indicate  the  correct  alternative  and  no  others.  For 
items  where  there  is  more  than  one  correct  alternative,  the  student  must 
get  80%  of  the  alternatives  correct.  That  is,  he  must  neither  mark 


incorrect  alternatives  (error  of  commission)  nor  fail  to  mark  correct 
alternatives  (error  of  omission)  on  more  than  20%  of  all  alternatives.1 


(2)  Free  Responses. 

(a)  A  free  response  is  where  a  student  is  free  to  answer 
a  written  test  item  in  his  own  words.  This  test  format  is  analogous  to 
an  "essay"  item  on  an  academic  test.  This  format  is  especially  appro¬ 
priate  when  the  task  involves  a  writing  skill,  such  as  filling  out  a 
form  or  making  a  report. 

(b)  The  most  serious  problem  with  free  response  test 
items  is  in  the  scoring  standards.  The  scoring  key  must  clearly  specify 
the  minimum  acceptable  answer.  Assuming  that  such  standards  can  be 
clearly  specified,  the  tester  still  faces  the  problem  of  deciphering 
answers  that  may  be  poorly  written,  both  in  terms  of  writing  style  and 
hand  writing.  And  because  of  the  relatively  longer  time  required  to 
grade  free  response  compared  to  recognition  response  tests,  the  former 
response  format  may  be  impractical  for  courses  having  large  enrollments. 

8-6.  Validate  Tests.  Compose  draft  versions  of  the  tests  and  determine 
whether  or  not  they  are  valid  tests  of  the  training  objectives. 

Ideally,  the  validation  process  is  conducted  by  actually  testing  both 
course  graduates  and  non-course  graduates.  Procedural  details  for  such 
a  full-scale  validation  process  are  described  in  Guidebook  for  Develop¬ 
ing  Criterion-Referenced  Test  (Swezey  &  Pearlstein,  1975)  and  Guidelines 
for  Development  of  Skill  Qualifications  Tests  (TRADOC,  1977).  As  argued 
in  Chapter  3,  however,  the  resources  required  to  test  a  large  sample  of 
soldiers  and  to  analyze  the  data  are  beyond  those  of  a  typical  course 
revision  project.  In  lieu  of  such  a  full-scale  validation,  an  accept¬ 
able  validation  procedure  is  to  have  experts  review  the  tests.  The 
following  represents  a  systematic  procedure  for  validating  test  using  a 
small  sample  of  experts. 

a.  Obtain  Experts.  At  least  five  experts  should  be  obtained  to 
review  the  course  in  question  and  have  significant  experience  in  the  job 
for  which  the  students  are  being  trained.  In  other  words,  experts 
should  have  the  same  MOS  as  course  entrants  but  in  a  higher  skill  level. 


1  The  exception  to  this  rule  is  where  the  student  can  pass  an  item  by 
either  marking  all  alternatives  or  failing  to  mark  any  of  the  alter¬ 
natives.  On  a  10-alternative  item,  for  example,  the  criterion  for 
passing  an  item  where  there  are  multiple  correct  answers  is  80%  or  8 
out  of  10  alternatives  correctly  marked  and  not  marked.  However,  if 
there  are  but  two  correct  answers,  the  student  could  pass  the  item  by 
not  marking  any  of  the  alternatives.  Conversely,  if  as  many  as  8  of 
the  10  alternatives  are  correct,  the  student  can  pass  the  item  by 
marking  all  of  the  alternatives.  In  cases  such  as  these,  you  must 
raise  the  criterion  (to  9  out  of  10  correct  in  both  examples)  to 
prevent  such  an  anomaly  in  scoring. 


Confirm  beforehand  that  all  experts  can,  in  fact,  perform  the  tasks  by 
reviewing  any  available  performance  information  (unit  tests,  SQT  scores, 
etc.)  or  talking  with  their  supervisors. 

b.  Written  Tests.  Have  all  experts  take  each  of  the  written 
tests.  Score  the  tests  immediately  and  discuss  errors  with  each  expert 
individually.  Specifically,  ask  experts  whether  or  not  each  recorded 
error  was  either  deserved  or  undeserved.  If  undeserved,  solicit  their 
ideas  on  making  the  test  more  accurate  or  fair.  Experts  may  even 
volunteer  errors  the  test  procedure  did  not  catch. 

c.  Hands-on  Tests.  Have  two  of  the  five  experts  act  as  testers. 
They  should  read  the  tester  instructions  to  understand  the  test  proce¬ 
dures.  Then  the  two  "testers"  should  use  the  Criterion  Scoring 
Checklists  to  independently  score  the  performance  of  the  remaining 
experts  as  they  take  the  test.  Both  testers  and  performers  should 
review  the  results  of  the  test,  again  paying  particular  attention  to 
errors.  In  addition,  you  should  examine  the  correspondence  between  the 
two  scorers  to  see  if  they  disagreed  in  scoring  anyone.  Be  sure  to 
determine  whether  the  disagreements  are  due  to  a  simple  lapse  in  atten¬ 
tion  or  to  ambiguity  in  the  scoring  procedure  that  would  indicate  the 
need  for  a  change. 

d.  Make  Changes.  Consolidate  the  suggestions  derived  from  the 
experts'  responses.  The  final  versions  of  the  tests  should  incorporate 
those  suggestions  that  make  the  tests  easier  to  administer  from  the 
tester's  point  of  view  and  fairer  to  take  from  the  student's  point  of 
view. 


8-7.  Example  Test.  To  demonstrate  how  the  test  documentation  is 
assembled,  an  example  test  is  presented  in  Figure  8-1  at  the  end  of  the 
chapter.  The  test  was  developed  for  a  19K  BNCOC  task  entitled  "Prepare 
and  Issue  Oral  Operation  Orders."  Much  of  the  test  is  self-explanatory. 
However,  the  following  points  require  some  elaboration: 

a.  According  to  the  proposed  organization  for  19K  BNCOC,  this 
task  is  located  in  the  cluster  entitled  "Prepare  Orders  and  Reports" 
along  with  two  tasks.  It  is  recommended  that  this  task  should  be 
combined  with  the  other  two  in  the  cluster  for  a  test  at  the  end  of  the 
block  of  instruction.  In  the  current  example,  however,  the  test  is 
presented  separately  to  simplify  the  exposition. 

b.  The  test  is  an  example  of  a  written  performance  test  with  a 
defineable  product,  namely  the  crew  operation  order.  The  test  also 
employs  a  free  response  format  in  order  for  the  student  to  compose  the 
operation  order.  The  free  response  affords  a  fairly  close  simulation  of 
actual  task  performance.  As  discussed  above,  the  free  response  format 
has  some  drawbacks.  The  problems  are  somewhat  offset  by  the  following 
considerations : 

(1)  The  scoring  of  free  format  tests  is  less  reliable  than 
that  for  the  recognition  format.  To  compensate,  the  end  product  is 
defined  as  completely  as  possible,  including  a  model  report  with  which 
to  compare  student  reports.  Secondly,  reliability  is  increased  by 
having  two  testers  score  the  student  reports  independently. 
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(2)  Free  format  tests  take  longer  to  score  than  recognition 
tests.  However,  because  19K  BNCOC  is  presently  limited  to  6-10  students 
per  cycle,  this  criticism  of  free  format  tests  is  less  critical. 

c.  One  obvious  difference  between  the  training  and  testing  objec¬ 
tive  is  that  the  soldier  is  trained  to  both  write  and  deliver  the  report 
orally  to  his  crew,  whereas  the  testing  criterion  requires  him  only  to 
write  the  crew  order.  There  are  two  reasons  for  the  discrepancy: 

(1)  The  written  report  provides  a  more  permanent  record  of 
performance  than  an  oral  report.  The  test  developer  can  surmount  the 
problem  of  the  relative  impermanence  of  the  oral  report  by  tape 
recording  student's  responses.  However,  this  procedure  was  deemed 
impractical  given  the  time  and  resource  constraints  of  19K  BNCOC. 

(2)  The  oral  part  of  the  report  is  easier  than  the  written 
part.  In  other  words,  if  a  tank  commander  can  compose  an  appropriate 
crew  order,  he  can  surely  deliver  it.  The  oral  part  of  the  task  may  be 
practiced  once  or  twice  during  training,  but  it  need  not  be  Included  in 
the  end-of-course  test. 


TEST  ADMINISTRATOR’S  GUIDE 
Prepare  and  Issue  Oral  Operation  Orders 
Task  No.  071-326-5626 


1.  EVALUATION  CRITERIA:  Given  a  pencil,  notebook,  appropriate  maps, 
and  recorded  platoon  leader's  orders,  the  tank  commander  must  write 
out  an  appropriate  oral  operation  order  for  his  crew  within  15  minutes 
after  receiving  the  platoon  leader's  orders. 

2.  EVALUATION  PROCEDURES:  Students  will  be  tested  in  a  group  in  a 
classroom  environment.  Two  testers  will  independently  score  the 
written  operation  orders  of  each  student  according  to  the  Criterion 
Scoring  Checklist.  Testers  should  compare  the  scores  of  each  student 
prior  to  providing  feedback.  Any  disagreements  between  testers  will 
be  resolved  by  a  brief  conference.  If  the  test  results  indicate  that 
a  student  has  not  achieved  the  testing  standard,  then  he  will  be  given 
an  explanation  of  his  error  and  a  chance  to  review  the  task  require¬ 
ments.  He  will  then  be  tested  a  second  time  on  the  entire  task  using 
another  recording  of  a  different  platoon  leader's  orders.  If  he  fails 
a  second  time,  he  will  be  counseled  by  testers  in  an  attempt  to 
identify  the  source  of  the  performance  problem.  Then  he  will  be  given 
a  third  test  trial  with  a  different  platoon  order.  Any  student 
failing  the  test  three  times  will  be  referred  to  course  administrators 
for  appropriate  action. 

3.  DIAGRAM  OF  THE  TESTING  FACILITY  N/A. 

4.  LIST  OF  PERSONNEL,  MATERIALS,  AND  EQUIPMENT  REQUIRED  FOR  THE  TEST: 

a.  Two  Criterion  Scoring  Checklists  per  student. 

b.  One  acetate-covered  strip  map  of  operations  area  and  grease 
pencil  per  student. 

c.  One  synchronized  slide  projector/tape  recorder  to  play  three 
prerecorded  platoon  orders  with  map  illustrations. 

d.  One  pencil  and  notebook  per  student. 

5.  TEST  PLANNING  TIME: 


a. 

Administrative  time 

10 

minutes 

b. 

Test  time 

15_ 

minutes 

c. 

Total  time 

25 

minutes 

Figure  8-1.  Example  Test  Documentation 
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6.  INSTRUCTIONS  TO  THE  SOLDIERS 


a.  Read  the  following  instructions  to  the  soldier,  exactly  as 
written. 

AT  THIS  STATION  YOU  WILL  BE  TESTED  ON  YOUR  ABILITY  TO  UNDERSTAND  A 
PLATOON  LEADER’ S  OPERATION  ORDER  AND  TO  WRITE  AN  APPROPRIATE 
OPERATION  ORDER  FOR  YOUR  CREW.  IN  THE  PRERECORDED  SCENARIO,  IT  IS 
2100  HOURS.  YOU  RECEIVED  A  WARNING  ORDER  FROM  YOUR  PLATOON  LEADER 
AN  HOUR  AGO,  AND  YOU  ARE  NOW  PREPARING  TO  RECEIVE  HIS  PLATOON 
OPERATION  ORDER.  I  AM  NOW  GOING  TO  PLAY  A  RECORDING  OF  HIS  ORDERS. 
LISTEN  CAREFULLY  AND  TAKE  NOTES.  I  AM  ONLY  GOING  TO  PLAY  THE 
RECORDING  ONCE.  AFTER  HEARING  THE  PLATOON  ORDER,  YOU  WILL  HAVE  10 
MINUTES  TO  WRITE  OUT  AN  APPROPRIATE  OPERATION  ORDER  FOR  YOUR  TANK 
CREW.  BE  SURE  TO  WRITE  YOUR  ORDERS  LEGIBLY  SO  THAT  WE  MAY  FAIRLY 
EVALUATE  IT.  REFER  TO  A  POINT  ON  THE  MAP  BY  MARKING  IT  ON  THE 
STRIP  WITH  A  NUMBER  AND  THEN  USING  THAT  NUMBER  IN  THE  REPORT.  ANY 
QUESTIONS? 

b.  The  platoon  orders  were  recorded  such  that  the  "platoon 
leader"  gives  the  orders  in  a  clear  and  distinct  fashion.  Play  the 
recorded  platoon  orders  once  for  the  students.  Only  under  extra¬ 
ordinary  circumstances  (e.g.,  very  loud  noises  that  might  mask  the 
recording)  should  you  replay  the  orders.  Testers  can  answer  any 
reasonable  questions  students  might  have  after  hearing  the  orders. 

Give  the  students  15  minutes  from  the  end  of  the  recorded  platoon 
orders  to  complete  writing  their  crew  orders. 

7.  SAFETY  PRECAUTIONS:  N/A. 

8.  ADDITIONAL  INFORMATION:  Although  the  students  listen  to  the 
platoon  orders  as  a  group,  they  should  work  on  their  crew  orders 
independently.  Both  testers  should  closely  monitor  them  during  the 
15  minute  period  while  they  compose  their  orders. 

9.  ANNEXES: 

a.  Criterion  Scoring  Checklist 

b.  Script  for  Platoon  Order 

c.  Model  of  Appropriate  Crew  Order 
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CRITERION  SCORING  CHECKLIST 
Prepare  and  Issue  Oral  Operation  Orders 
Task  No.  071-326-5626 

Name  of  Student  _  Date  __ 

Evaluator  _ 


_ Evaluation _ 

First  Second  Third 
NO  NO  NO 

GO  GO  GO  GO  GO  GO  Comments 


DID  THE  STUDENT'S  CREW 
ORDERS  INCLUDE  EACH  OF  THE 
FOLLOWING  PARAGRAPHS: 

1.  SITUATION 

a.  Enemy  Forces 

b.  Friendly  Forces 

2.  MISSION 

3 .  EXECUTION 

a.  Scheme  of  Maneuver 

b.  Fire  Support 

c.  Specific  Instructions 

d.  Coordinating  Instruc¬ 
tions 

A.  SERVICE  SUPPORT 
5.  COMMAND  AND  SIGNAL 

a.  Communications 

b.  Succession  of  Command 


Figure  8-1.  Example  Test  Documentation  (Continued) 
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_ Evaluation _ 

First  Second  Third 
NO  NO  NO 

GO  GO  GO  GO  GO  GO  Comments 


DID  STUDENT'S  ORDERS  INCLUDE 
AN  ACCURATE  REPORT  OF  THE 
ESSENTIAL  POINTS  IN  EACH  OF 
THE  FOLLOWING  PARAGRAPHS 
(SEE  UNDERLINED  PORTIONS  OF 
APPENDIX  C): 

1.  SITUATION 

a.  Enemy  Forces 

b.  Friendly  Forces 

2.  MISSION 

3 .  EXECUTION 

a.  Scheme  of  Maneuver 

b.  Fire  Support 

c.  Specific  Instructions 

d.  Coordinating  Instruc¬ 
tions 

4.  SERVICE  SUPPORT 

5.  COMMAND  AND  SIGNAL 

a.  Communications 

b.  Succession  of  Command 


Figure  8-1.  Example  Test  Documentation  (Continued) 
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CRITERIA  FOR  PASSING  TEST 


To  receive  a  GO,  the  student  must  receive  a  GO  on  all  items  and 
complete  the  task  within  15  minutes.  He  may  not  be  evaluated  more 
than  three  times. 


First 

Attempt 

NO 

GO  GO 


Second 

Attempt 

NO 

GO  GO 


Third 

Attempt 

NO 

GO  GO 


OVERALL  EVALUATION 


DID  YOUR  EVALUATION  OF  THE  STUDENT 

AGREE  WITH  YOUR  PARTNER'S?  YES  NO 

IF  NO,  HOW  WAS  THE  DISAGREEMENT 
RESOLVED? 


SIGNATURES 

Evaluator  Soldier 


ANNEX  B 


Platoon  Operation  Orders 


1.  SITUATION 

a.  ENEMY:  Enemy  forces  are  withdrawing  east  and  appear  to  be 
occupying  the  far  bank  of  the  Green  River.  We  can  expect  platoon 
size  recon  units  to  try  to  delay  our  advance  toward  Green  River. 
Remember  last  week  the  enemy  hit  us  with  persistent  chemical  agents 
when  we  moved  through  Dexter.  The  terrain  we'll  be  moving  over  con¬ 
sists  of  rolling  hills,  scattered  trees,  and  in  some  areas,  fields 
of  fire  to  3000  meters.  The  only  obstacle  is  the  Salt  River,  which 
is  believed  to  be  fordable  in  some  places.  Weather  is  expected  to  be 
clear  and  dry  with  winds  at  10  knots  from  the  west. 

b.  FRIENDLY  FORCES:  Our  team  will  cross  the  SP  at  this  location 
(indicate  as  Point  1  on  the  slides  of  the  map)  at  0600  hours  tomorrow 
morning,  passes  through  the  1/3  Cavalry  at  this  point  (Point  2),  and 
moves  east  along  Highway  N4  to  seize  Hill  609,  located  here  (Point  3). 
Our  platoon  will  lead,  followed  by  the  team  command  group,  the  3d 
Infantry  Platoon,  and  the  2d  Tank  Platoon.  Another  team,  with  a 
similar  mission  will  be  moving  east  along  Highway  N2,  5,000  meters  to 
our  left  flank.  Our  right  flank  will  be  open. 

2.  MISSION:  Our  platoon  crosses  the  SP,  moves  through  the  cavalry 
crossing  point,  and  leads  the  team  in  a  movement  to  contact  to  seize 
Hill  609. 

3.  EXECUTION: 

a.  SCHEME  OF  MANEUVER:  We  will  move  past  the  SP  and  through  the 
crossing  point  in  a  column  formation;  once  clear  of  the  cavalry  posi¬ 
tion  we  will  change  to  a  combat  column  formation.  Any  other  changes 
in  formation  will  depend  upon  the  tactical  situation  and  the  terrain. 
Whenever  possible  we  will  bypass  enemy  positions  because  speed  in 
reaching  Hill  609  is  essential. 

b.  FIRE  SUPPORT:  Our  platoon  will  initially  have  priority  of 
fires  within  the  team. 

c.  SPECIFIC  INSTRUCTIONS:  When  approaching  and  passing  through 
the  cavalry  crossing  point,  main  guns  will  be  at  maximum  elevation. 

As  soon  as  we  get  into  the  combat  column  formation  beyond  the  cavalry 
position,  main  guns  will  be  at  normal  elevation.  All  formations  and 
reaction  drills  will  be  in  accordance  with  the  unit  tactical  SOP.  Be 
watching  for  signals  from  tl»e  platoon  leader  or  the  platoon  sergeant, 
and  remember  to  maintain  your  position  in  each  formation. 


Figure  8-1.  Example  Test  Documentation  (Continued) 
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d.  COORDINATING  INSTRUCTIONS:  Movement  will  be  in  MOPP  1. 
Standto  0400,  breakfast  0415,  check  assembly  area  0515,  crews  mounted 
0540,  and  depart  assembly  area  0550,  cross  SP  0600. 

4.  SERVICE  AND  SUPPORT:  Our  unit  tactical  SOP. 

5.  COMMAND  AND  SIGNAL: 

a.  SIGNAL:  CE0I  Index  1,  Edition  B  is  in  effect.  Listening 
silence  is  in  effect  until  enemy  contact.  Two  green  star  clusters  is 
the  emergency  signal  for  shifting  or  lifting  supporting  or  suppres¬ 
sive  fires. 

b.  COMMAND:  I  will  be  in  my  usual  platoon  formation  position. 
Succession  of  command  is  platoon  sergeant,  TC  tank  2,  TC  tank  4. 

Time  now  is  2130  hours.  Are  there  any  questions? 

Figure  8-1.  Example  Test  Documentation  (Continued) 
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ANNEX  C 


Model  Crew  Operation  Orders 


1.  SITUATION: 

a.  ENEMY:  Enemy  forces  are  withdrawing  to  the  east  and  are 
covering  the  withdrawal  with  recon  platoon  size  delaying  forces  along 
Highway  N4.  The  enemy  has  recently  used  persistent  chemical  agents 
against  us.  The  terrain  in  the  area  consists  of  rolling  hills,  ~ 
scattered  trees,  and  some  fields  of  fire  to  3000  meters.  The  only 
obstacle  short  of  Hill  609  is  the  Salt  River,  which  appears  to  be 
fordable  in  some  places.  The  weather  will  be  clear  and  dry  with 
westerly  winds  at  10  knots. 

b.  FRIENDLY  FORCES:  1st  Platoon  will  lead  the  team  across  the 
SP  located  here  (indicated  as  Point  1  on  the  strip  map)  at  0600  hours 
tomorrow,  passes  through  the  1/3  crossing  point  located  here 

(Point  2),  and  moves  east  along  Highway  N4  to  seize  Hill  609  located 
here  (Point  3).  The  1st  Platoon  will  be  followed  by  the  team  command 
group,  the  3d  Infantry  Platoon,  and  the  2d  Tank  Platoon.  Another 
team  will  move  east  along  Highway  N2  5,000  meters  to  our  left  flank. 
Our  right  flank  will  be  open. 

2.  MISSION:  Our  mission  during  the  operation  is  to  maintain  the 
wingman  position  during  movement  and  reaction  formations,  to  respond 
to  all  platoon  leader  or  platoon  sergeant  orders,  to  destroy  enemy 
targets  as  acquired,  and  to  submit  appropriate  reports. 

3.  EXECUTION: 

a  SCHEME  OF  MANEUVER:  We  will  maintain  our  position  in  column 
formation  until  the  platoon  clears  the  cavalry  crossing  point.  At 
that  time,  the  platoon  will  move  into  a  combat  column  formation,  and 
we  will  maintain  our  wingman  position  in  that  formation. 

b.  FIRE  SUPPORT:  Our  platoon  has  priority  of  indirect  fires. 

c.  SPECIFIC  INSTRUCTIONS:  When  we  approach  and  pass  through  the 
cavalry  crossing  point,  the  main  gun  will  be  at  maximum  elevation. 
Beyond  the  crossing  point,  the  main  gun  will  be  depressed  to  normal 
elevation.  All  formation  and  reaction  drills  will  be  per  the  unit 
tactical  SOP.  The  loader  and  driver  will  be  alert  for  platoon  leader 
and  platoon  sergeant  hand  and  arm  signals. 

d.  COORDINATING  INSTRUCTIONS:  Movement  will  start  in  MOPP  1. 
Standto  0A00,  breakfast  0415,  assembly  area  checked  0515,  crew 
mounted  0540,  depart  assembly  area  0550,  and  cross  SP  at  0600. 


Figure  8-1.  Example  Test  Documentation  (Continued) 
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4.  SERVICE  SUPPORT:  Per  unit  tactical  SOP. 

5.  COMMAND  AND  SIGNAL: 

a.  SIGNAL:  CEOI  Index  1,  Edition  B  is  in  effect  with  listening 
silence  in  effect  until  enemy  contact.  Two  green  star  clusters  is 
the  emergency  signal  for  shifting  or  lifting  supporting  or  suppres¬ 
sive  fires. 

b.  COMMAND:  Succession  of  command  is  gunner,  driver,  loader. 
Time  now  is  2200  hours.  Are  there  any  questions? 


Figure  8-1.  Example  Test  Documentation  (Continued) 
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OUTLINE  TRAINING  PROGRAM 


9-1.  Introduction.  Up  to  this  point  in  the  process  of  course  revision, 
you  have  probably  done  most  of  the  work  yourself  or  with  one  or  two 
assistants.  You  are  now  about  to  enter  the  development  phase  which  will 
probably  require  the  help  of  many  more  people.  Prior  to  starting  into 
the  labor-intensive  development  phase,  you  should  summarize  the  design 
of  the  revised  training  program  and  obtain  the  approval  of  training 
management . 

9-2.  Prepare  Outline.  The  outline  should  present  the  work  performed  up 
to  this  point  in  an  organized  fashion.  In  a  sense,  the  course  outline 
resembles  a  POI.  However,  the  POI  is  usually  developed  after  all  train¬ 
ing  materials  have  been  produced  and  is  therefore  more  detailed.  In 
contrast,  the  following  outline  is  designed  to  present  the  course  only 
at  the  present  stage  of  revision. 

a.  Overview.  The  first  part  of  the  suggested  outline  presents  a 
general  description  of  the  course.  It,  in  turn,  consists  of  the 
following  two  subcomponents: 

(1)  Course  Objective.  The  outline  should  start  with  a 
statement  of  the  overall  course  objective  as  developed  early  in  the 
revision  process  (Chapter  2). 

(2)  Prerequisites.  From  information  given  in  the  description 
of  course  entrants  (Chapter  3),  derive  a  reasonable  list  of  requirements 
for  students  entering  the  course.  In  addition  to  any  special  entrance 
requirement  (performance  certification,  precourse  test  scores,  etc.), 
the  following  elements  should  also  be  included  in  the  statement: 

(a)  minimum  and  maximum  grade, 

(b)  military  courses  or  schools  from  which  the  entrant 
must  have  graduated, 

(c)  minimum  length  of  service  or  minimum  remaining 
service  obligation, 

(d)  whether  or  not  a  security  clearance  is  required. 

b.  Course  Summary.  The  course  summary  consists  of  every  element 
of  the  course  and  the  corresponding  time  requirements.  The  time 
requirements  are  all  estimates,  but  they  should  give  training  management 
some  rough  idea  of  course  length.  Construct  the  summary  according  to 
the  following  procedures: 

(1)  Identify  All  Course  Elements.  There  are  at  least  three 
types  of  course  elements  you  need  to  identify: 


(a)  the  individual  blocks  of  instruction  within 
functional  areas, 

(b)  any  field  exercises  needed  to  support  and  reinforce 
classroom  instruction,  and 

(c)  noninstructional  elements  that  require  significant 
amounts  of  course  time.  Example  noninstructional  elements  include: 

1^  testing  activities, 

2_  in-  and  out-processing, 

3^  commandant's  time,  and 

4_  physical  training. 

(2)  Identify  Less  Critical  Elements.  You  may  wish  to  present 
management  with  outlines  for  alternative  courses  which  differ  in  length. 
Use  the  information  obtained  in  the  task  prioritization  process 
(Chapter  4)  to  develop  a  rationale  for  excluding  certain  tasks.  For 
instance,  one  alternative  might  be  a  full  course  which  includes  all 
"must  train"  and  "should  train"  tasks.  (Of  course,  "don't  train"  tasks 
should  not  appear  on  any  course  outline.)  As  an  alternative,  you  might 
also  outline  a  shorter  version  of  the  course  which  only  includes  "must 
train"  tasks.  Other  alternatives  might  include  the  most  critical 
"should  train"  tasks  as  well. 


(3)  Organize  Elements.  Organize  the  instructional  elements 
according  to  the  structure/sequence  determined  earlier  (Chapter  7).  To 
this  add  the  noninstructional  elements  identified  above  and  any  field 
exercises  that  are  necessary  supplements  to  classroom  training.  These 
latter  elements  should  be  inserted  at  the  appropriate  point  in  the 
structure/sequence  of  the  course. 


(4)  Calculate  Time  Requirements.  Calculate  the  total  number 
of  hours  for  the  "complete"  course  and  any  other  shortened  courses  to 
which  you  wish  to  compare  and  contrast.  Proceed  as  follows: 


(a)  Estimate  the  number  of  hours  required  for  each  block 
of  instruction.  Wherever  possible,  base  these  estimates  on  times 
presented  in  the  existing  POI.  Ensure  that  instructional  blocks  are 
consistent  with  an  eight-hour  training  day  (e.g.,  two  related  blocks  of 
instruction  could  be  trained  in  two,  four-hour  periods  on  the  same  day). 

(b)  Calculate  the  total  time  requirement  at  least  three 

ways: 

1_  total  number  of  hours, 

2_  total  number  of  days  assuming  eight  hours/day, 

and 


3^  total  number  of  weeks  assuming  five  days/week. 


(5)  Example  Course  Summary.  Figure  9-1  presents  an 
example  of  a  course  summary  taken  from  19K  BNCOC.  Several  course 
elements  have  been  added  as  shown  by  the  footnotes.  Detailed 
descriptions  of  these  elements  are  presented  In  Morrison,  Drucker,  and 
O'Brien  (1985).  The  presentation  compares  the  hours  required  for  the 
complete  course,  which  should  last  about  nine  weeks,  to  a  shorter 
course,  which  should  take  about  six  weeks  to  complete.  The  shorter 
course  was  derived  by  eliminating  the  "should  train"  tasks  and 
eliminating  the  Country  Fair  Exercise. 


Hours  of 

Instruction 

Course  Cluster/Activity 

6-Week 

Course 

9-Week 

Course 

In  Processing1 

8 

8 

Precourse  Diagnostic  Tests1 

8 

8 

Leadership 

8 

20 

Leadership  Reaction  Course2 

4 

4 

Training  Procedures 

22 

40 

NBC 

4 

20 

Mine  Warfare 

0 

4 

Communicat ions 

8 

16 

Land  Navigation 

18 

32 

Land  Navigation  Pathfinder  Course2 

8 

8 

Maintenance 

12 

16 

Country  Fair2 

0 

4 

Tactics 

20 

52 

Single  Tank  Tactical  Exercise2 

16 

16 

Intra-platoon  Exercise2 

32 

32 

End-of -Course  Test1 

8 

8 

Out  Processing1 

8 

8 

Totals 

240 

360 

^oninstructional  elements  added  to  course. 
2Field  exercises  added  to  course. 

Figure  9-1.  Example  Course  Summary 


c.  Functional  Area  Annexes. 


(1)  The  functional  area  annexes  are  the  most  detailed  account 
of  the  course  in  that  they  list  the  training  objectives  for  each  block 
and  tasks  within  blocks  for  each  functional  area.  The  objectives  are 
all  listed  in  the  correct  order  within  the  functional  area,  and  the 
annexes  corresponding  to  functional  areas  are  attached  in  the 
appropriate  sequence. 

(2)  The  following  example  (Figure  9-2)  is  a  portion  of  the 
Tactics  functional  area  annex  that  corresponds  to  the  block  of  instruc¬ 
tion  on  preparing  orders  and  reports. 


I 


CLUSTER:  Prepare  Orders  and  Reports 

OBJECTIVE :  The  tank  commander  must  prepare  and  issue  appropriate 
reports  in  a  timely  and  accurate  manner. 

TOTAL  HOURS:  8 

TASK  1 :  Prepare  and  Issue  an  Oral  Operation  Order 

OBJECTIVE:  Given  a  pencil,  notebook,  appropriate  maps,  and  a 
warning  order  to  standby  for  an  order;  the  tank  commander  must 
analyze  a  platoon  leader’s  operation  order,  write  out  an 
appropriate  and  accurate  crew  operation  order,  and  deliver  it  to 
his  crew  within  15  minutes  of  receiving  the  platoon  leader's 
orders. 

HOURS :  4  (2  lecture/demonstration,  2  practical  exercise) 

TASK  2 :  Prepare  a  Situation  Report  (SITREP) 

OBJECTIVE:  Given  a  tactical  situation  that,  according  to 
platoon  tactical  SOP,  requires  a  situation  report  to  be 
submitted;  submit  within  10  minutes  an  oral  situation  report  by 
radio  or  hot  loop  to  the  platoon  leader.  The  report  must 
include  all  information  in  appropriate  paragraphs  of  a  standard 
situation  report;  locations  and  quantities  must  be  encrypted; 
and  brevity  list  of  CEOI  must  be  used. 

HOURS :  2  (1  lecture/demonstration,  1  practical  exercise) 

TASK  3 :  Prepare  and  Submit  a  Standard  Shelling,  Mortaring  and 
Bombing  Report 

OBJECTIVE:  Given  a  report  of  either  shelling,  mortaring  or 
bombing  and  a  STANAG  2008  format,  the  tank  commander  must  record 
the  Information  received  on  a  STANAG  2009  (DA  Form  2185-R)  and 
send  the  information  to  the  next  higher  headquarters  by  the  most 
rapid  means  (radio,  telephone,  or  messenger)  and  encode  as 
necessary. 

HOURS :  2  (1  lecture/demonstration,  1  practical  exercise) 

Figure  9-2.  Example  Portion  of  Functional  Area  Annex 
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9-3.  Present  to  Management.  The  outline  of  the  course  should  be 
presented  to  management  for  formal  approval.  In  a  cover  letter  to  the 
outline,  present  management  with  the  following  issues: 

a.  Introduction .  Introduce  the  outline  by  describing  its 
purpose.  Be  sure  to  emphasize  its  preliminary  character. 

b.  Method.  Present  a  brief  summary  of  the  method  that  you  used 
to  develop  the  course  outline.  If  you  are  presenting  alternative 
courses,  be  sure  to  include  the  method  used  to  derive  each  version. 

c.  Development  Costs.  Estimate  of  time  and  personnel  costs 
associated  with  development  phases. 

d.  Equipment  Costs.  Provide  estimated  costs  of  any  training 
equipment  that  is  not  in  the  inventory  of  the  existing  course. 

e.  Instructor  Requirements.  Provide  your  best  guess  at  the 
number  of  instructors  that  are  required  to  implement  the  revised  course. 

f*  Length  of  Course.  If  alternative  courses  are  compared/con¬ 
trasted,  be  sure  to  describe  how  the  alternatives  were  derived.  Solicit 
management's  opinion  on  the  best  tradeoff  between  the  comprehensiveness 
of  the  course  and  the  costs  in  terms  of  time  and  resources. 
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task  analysis  Is  too 
expensive  to  perforn  prior 
to  task  selection. 
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